Skip to content

# L7a: Autoresearch Loop ​

Build a self-improving agent with integrity guards.

Module: M7 Advanced Topics
Est. Time: 75 min
Files: `starter.py`, `solution.py`

## Objective ​

Create an agent that runs experiments, measures its own performance, logs results, and decides whether to keep or discard each change.

## Concepts ​

- Autoresearch: run → measure → log → decide
- Code hashing to detect grinding
- Median vs best comparison
- Reward hack defense

## Starter ​

bash

    cd course/labs/L7-autoresearch/
    python starter.py

## Solution ​

bash

    python solution.py

## Experiment Log Format ​

jsonl

    {"run": 1, "status": "baseline", "metric": {"name": "latency", "value": 52, "unit": "ms"}}
    {"run": 2, "status": "keep", "metric": {"name": "latency", "value": 46}, "deltaPct": -11.5}
    {"run": 3, "status": "discard", "metric": {"name": "latency", "value": 53}, "deltaPct": +1.9}

## Integrity Guards ​

Threat | Detection | Prevention
--- | --- | ---
Grinding (same code re-run) | Code hash comparison | Skip run
Noise-chasing | Median vs best comparison | Use median, not best
Reward hacking | Timing function isolation | Verify computation not shortcut
Test set leakage | Test data hash verification | Assert data unchanged