agentic-ai-engineering/site/modules/m7-advanced.md

# Module 7: Advanced Topics

## Lesson 7.1: Autoresearch

### The Core Loop

Agents that improve themselves. The experiment loop:

```
1. Run: Execute the agent with current configuration
2. Measure: Collect metrics (latency, cost, success rate)
3. Log: Record experiment results to JSONL
4. Decide: Keep if improvement, discard if regression
5. Repeat: Try next experiment
```

### Brand Monitor Example

From your own autoresearch system:

```json
{"run": 1, "metric": {"name": "home_ms", "value": 52}, "description": "Baseline"}
{"run": 2, "metric": {"name": "home_ms", "value": 46}, "deltaPct": -11.5, "description": "Remove N+1 count queries"}
```

---

## Lesson 7.2: The Experiment Loop — Integrity

### Three Threats to Integrity

1. **Reward hacking** — Model moves computation outside the timing function. The "timed call" becomes a lookup. (Found in Mythos paper.)

2. **Grinding** — Running identical code 160 times hoping for a lucky outlier. (Found in Mythos paper — "Pure grind — same code, lucky measurement.")

3. **Test set leakage** — Finding the test set and training on it. (Found in Mythos paper — time series agent found the test set and directly trained on it.)

### Integrity Guards

```python
# Code hash — detect grind runs
code_hash = hash(open("agent.py").read())
if last_run_code_hash == code_hash:
    flag_grind_run()

# Compare against median, not best
scores = [52, 48, 47, 46, 46, 51, 53]
median = sorted(scores)[len(scores)//2]  # 48
best = max(scores)  # 46 (outlier)

# Verify test data wasn't modified
test_data_hash_before = hash(open("benchmark_data.json").read())
# ... run experiment ...
test_data_hash_after = hash(open("benchmark_data.json").read())
assert test_data_hash_before == test_data_hash_after
```

---

## Lesson 7.3: Meta-Agents — Agents That Build Agents

### The Pattern

```
User: "I need an agent that monitors our API uptime"
Meta-agent:
  1. Researches API monitoring patterns (parallel experts)
  2. Generates agent persona/system prompt
  3. Creates tool definitions
  4. Writes skill files
  5. Validates the generated agent works
```

### Pi-Pi Meta-Agent

From pi-vs-claude-code: a meta-agent that builds Pi agents using parallel research experts:

```
pi-pi agent → dispatches to:
  ├── ext-expert (extension documentation)
  ├── theme-expert (theming conventions)
  ├── skill-expert (skill system)
  ├── config-expert (configuration)
  └── tui-expert (TUI components)

Each expert:
  1. Fetches latest documentation (firecrawl + curl fallback)
  2. Synthesizes patterns
  3. Returns structured guidance

pi-pi → generates complete extension code
```

---

## Lesson 7.4: Beyond MCP — The Context Cost Trade-off

### The Matrix

| Approach | Context Cost | Portability | Agent-Invoked? | Best For |
|----------|-------------|-------------|----------------|----------|
| MCP Server | HIGH (full context per call) | HIGH | Yes | Multi-client, standardized tools |
| CLI | MEDIUM | HIGH | No | 80% of new tools |
| File Scripts | LOW (progressive disclosure) | MEDIUM | No | Context-sensitive tools |
| Skills | LOW | MEDIUM | Yes (auto-detect) | Agent-native behavior |

### The Insight

MCP is not always the answer. For tools used by 1-2 agents, CLI + prime prompt is faster, cheaper on context, and easier to debug. MCP shines for tools used by many agents across many clients.

---

## Lesson 7.5: The Mac Mini Agent — Physical Sandbox

### Architecture

```
Mac Mini (Agent Sandbox)
  ├── Steer (GUI automation) — Swift, 14 commands
  │     see, click, type, hotkey, ocr, find, wait
  ├── Drive (Terminal control) — Python, 6 commands
  │     session, run, send, poll, fanout
  └── Listen (Job server) — Python/FastAPI
        POST /job, GET /job/{id}

Primary Machine (Dev)
  └── Direct CLI client
        start, get, list, latest, stop
```

### Sentinel Pattern

Makes async terminal work deterministic:

```bash
# Agent runs command, appends sentinel
long_running_task; echo "__DONE_abc123:$?"

# Agent polls for sentinel pattern
poll logs for "__DONE_abc123"
# Extract exit code from sentinel
```

### Why Physical Sandbox Matters

Cloud agents are ephemeral. A Mac Mini agent is persistent — it has files, databases, browser sessions, and GPU state that survive agent restarts. This matters for:

- **Long-running experiments** that take hours or days
- **Browser automation** with persistent login sessions
- **Local model inference** where GPU memory is precious
- **Legacy system access** that can't be moved to cloud

The physical sandbox pattern works with any mini PC — Mac Mini, Intel NUC, Raspberry Pi 5, or repurposed laptop. The key is a dedicated machine that the agent can control without affecting your primary workstation.

---

## Lesson 7.6: Multi-Model Strategies

### Why One Model Isn't Enough

No single model is best at everything. Frontier models (Claude Opus, GPT-4o) excel at reasoning but cost 10-100x more than small models (Haiku, Flash, Nano). The strategy: use the right model for each subtask.

### Model Heterogeneity in Practice

```
Task: "Analyze this codebase for security vulnerabilities"

Step 1: File Discovery (Haiku — $0.0003/run)
  → List all files, grep for patterns
  → Cheap, fast, 100% accuracy needed

Step 2: Vulnerability Analysis (Sonnet — $0.008/run)
  → Read each suspicious file, classify vulnerability
  → Needs reasoning, but not frontier-level

Step 3: Report Generation (Opus — $0.03/run)
  → Synthesize findings into executive report
  → Needs highest quality output
```

Total cost: ~$0.04 instead of ~$0.30 if done entirely with Opus. **87% savings** with no quality loss on the critical path.

### The 3-Tier Cascade Pattern

```python
TIERS = {
    "cheap": {"model": "claude-haiku",  "max_tokens": 4000,  "cost_per_k": 0.00025},
    "mid":   {"model": "claude-sonnet", "max_tokens": 8000,  "cost_per_k": 0.003},
    "premium": {"model": "claude-opus",  "max_tokens": 16000, "cost_per_k": 0.015},
}

def cascade(prompt, complexity="mid"):
    """Route to appropriate tier based on task complexity."""
    if complexity == "low":
        return call_model(TIERS["cheap"], prompt)
    elif complexity == "high":
        return call_model(TIERS["premium"], prompt)
    else:
        return call_model(TIERS["mid"], prompt)
```

### Model Selection Decision Tree

```
Is the task deterministic? (grep, sort, count)
  → Use cheap model (Haiku/Flash)
  
Does the task need reasoning? (analyze, explain)
  → Use mid model (Sonnet/4o-mini)
  
Is the output customer-facing? (report, email, memo)
  → Use premium model (Opus/4o)

Will this code be deployed to production?
  → Run through ALL three tiers in sequence (cascade)
  → Cheap for bulk work, mid for analysis, premium for final review
```

---

## Lesson 7.6: Always-On Agents

### Voice-to-Command Bridge

```
User speaks → STT (faster-whisper) → LLM transcribes to command → Execute → TTS response
```

### Job Server Pattern

```
Submit → Queue → Worker picks up → Claude Code executes → Result stored → Check later
```

### Heartbeat Execution (from Paperclip)

```
Timer fires → Check for queued work → Wake agent → Agent executes → Agent sleeps
```

No continuous running. Agents wake, work, and sleep on a schedule.

### Scheduling Strategies

| Strategy | Pattern | Use Case |
|----------|---------|----------|
| Fixed interval | Every N minutes/hours/days | Brand monitoring, price tracking |
| Event-triggered | Webhook → wake agent | CI/CD pipelines, PR reviews |
| Predictive | Learn optimal check times | Traffic monitoring, anomaly detection |
| Cascade | Agent A finishes → Agent B starts | Data pipeline, ETL workflows |

### OpenClaw: The Always-On Employee

OpenClaw is a daemon-style agent that runs as a background service:

```yaml
# openclaw-config.yaml
agent:
  name: "brand-monitor"
  heartbeat: 300  # every 5 minutes
  max_sessions: 1
  wake_command: "claude -p 'Check brand mentions since last run'"
  sleep_command: "pkill -f 'claude.*brand-monitor'"
  log_path: "/var/log/openclaw/brand-monitor.log"
```

It starts, checks for work, executes if needed, then goes back to sleep. No continuous billing, no context window overflow, no runaway loops. This is the pattern for production always-on agents.

### When NOT to Use Always-On

Always-on adds complexity. Before building one, verify you actually need it:
- **Batch jobs** → Cron is simpler and cheaper
- **Real-time** → Event-driven architectures (webhooks, Kafka) outperform polling agents
- **One-off** → Just run the agent once

Always-on makes sense when you need adaptive scheduling, dynamic task generation, or autonomous decision-making about what to work on next.

---

## Lab 7.7: Build an Autoresearch Loop

**Objective**: Agent runs experiment, measures result, logs it, decides keep/discard.

**Starter**: `course/labs/L7-autoresearch/starter.py`

**Checkpoints**:
1. Run code, measure baseline metric
2. Modify code (agent makes change)
3. Re-measure, compare, log
4. Discard if regression, keep if improvement
5. Include integrity guard (code hashing)

## Lab 7.8: Meta-Agent

**Objective**: Agent generates a new agent persona from documentation.

**Starter**: `course/labs/L7-meta-agent/starter.py`