Definition: An AI agent = LLM + Tools + Loop. Without any one of these three, it's not an agent.
Agent = LLM (reasoning engine)
+ Tools (capability surface)
+ Loop (autonomous decision cycle)LLM — The reasoning engine. Given context + available tools, it decides which tool to call and with what parameters. The LLM is NOT the agent — it's the brain of the agent.
Tools — The capability surface. Functions the agent can call: read files, run commands, search the web, query databases, call APIs. Each tool has a name, description, and input schema.
Loop — The autonomous decision cycle. Think (LLM decides) → Act (tool executes) → Observe (result comes back) → Repeat. The loop is what makes it autonomous.
Core insight: The agent harness IS the product, not the model.
Most engineers focus on model quality ("which model is best?"). The data tells a different story:
| Aspect | Model Matters | Harness Matters |
|---|---|---|
| Tool selection accuracy | ✓ | ✓ (prompt design) |
| Loop termination | ✗ | ✓ (iteration limits) |
| Security | ✗ | ✓ (hooks, sandboxing) |
| Cost | ✓ (per-token price) | ✓ (how many calls made) |
| Reliability | ✗ | ✓ (retry, fallback, verify) |
| Context management | ✗ | ✓ (window, summarization) |
The multiplier effect: A good harness with a mediocre model outperforms a bad harness with the best model. The harness determines how many tokens you spend, how safely you operate, and how reliably you recover from failure.
A harness is not a single thing. It's five subsystems working together:
HARNESS
├── Instructions — System prompts, AGENTS.md, CLAUDE.md, skills
├── Tools — Bash, file ops, MCP servers, APIs, custom tools
├── Environment — File system, runtime, git worktree, sandbox
├── State — Session history, context window, mental models
└── Verification — Tests, evals, verifier agents, review gatesEvery agent failure maps to one of these five. When something breaks, don't swap the model — check each subsystem in order.
Your control over an agent boils down to four dimensions:
| Dimension | What You Control | Leverage |
|---|---|---|
| Context | What the agent knows (files, instructions, history) | High |
| Model | Which LLM powers the agent | Low (same for everyone) |
| Prompt | How you instruct the agent (system prompt, tools, skills) | Very high |
| Tools | What the agent can do (capability surface) | Highest |
Great harness engineering = managing these four dimensions across many agents at scale.
This is the central question of agentic engineering. Every architectural decision — security, verification, monitoring, autonomy — comes back to trust. The more trust you build into your systems, the more you can delegate. The more you delegate, the more leverage you have.
Not everyone agrees. George Hotz (founder of comma.ai, first iPhone jailbreaker) argues that AI agents cannot program and their adoption will be one of the most costly mistakes in the field's history.
His core critique:
"Golden era for buckets of slop, dark age for gems of quality." — George Hotz
If you only hear the optimistic view, you'll deploy agents naively and get burned. Hotz's critique identifies real failure modes. This course addresses each one:
| Hotz's Critique | Our Solution | Module |
|---|---|---|
| Slop amplification | CI/CD quality gates + golden datasets | M5 |
| Hard-to-detect breakage | Verifier pattern (read-only checks) | M3 |
| Slot machine polish problem | Defense-in-depth + confidence ladder | M3 |
| No world models | Evaluation frameworks + pass@k | M6 |
The honest take: Agents produce slop. The harness, verification, and production patterns are what turn slop into shipped quality. Without the harness, Hotz is right.
Armin (creator of Flask, Pi maintainer) reveals that Pi is built with Pi — agents building the tools that build agents. This is the meta-agent pattern in production. But he's honest about limitations:
"We're quite far off today from where Bun and [others] are. Today it does not seem like we know how to pull off a dark factory."
The honest middle ground between full automation and pure skepticism.
Do you trust your agents? The answer should be: yes, because you've engineered the trust. Security (M3), verification (M3), production patterns (M5), and evaluation (M6) are how you earn that trust.
This is the central question of agentic engineering. Every architectural decision — security, verification, monitoring, autonomy — comes back to trust. The more trust you build into your systems, the more you can delegate. The more you delegate, the more leverage you have.
Uncertainty level: Can you write deterministic code for this?
Failure cost: What happens if the agent messes up?
Loop depth: How many decisions must the agent make?
Observability requirement: Can you tolerate a black box?
Can you solve it deterministically?
├── YES → Write code. Don't use an agent.
└── NO →
Is failure cost HIGH?
├── YES → Agent + human-in-the-loop + verification
└── NO →
Is loop depth > 10?
├── YES → Multi-agent system with orchestration
└── NO → Simple agent loop with tool useCore principle: All necessary context for your agent should live in the repository. The repo is the single source of truth — not a README, not a wiki, not tribal knowledge.
repo/
├── AGENTS.md or CLAUDE.md — Agent instructions (THE file)
├── .cursorrules — Tool-specific rules
├── skills/ — Reusable skill definitions
│ ├── skill-name/
│ │ ├── SKILL.md — What the skill does
│ │ └── scripts/ — Supporting scripts
├── .mcp.json — MCP server configuration
├── .claude/hooks/ — Lifecycle hooks
├── init.sh — Environment setup script
├── feature_list.json — Tracked feature inventory
└── tests/ — Verification evidenceAn agent can only see what you put in front of it. A well-structured repo:
Rule: If an agent needs to know something to do its job, that information must be in a file in the repo.
Every agent, regardless of complexity, follows this loop:
while not done and iterations < MAX_ITERATIONS:
1. THINK: LLM receives context + tool list → decides what to do
2. ACT: Tool executes (read file, run command, query DB, etc.)
3. OBSERVE: Result comes back as a message
4. REPEAT or DONE: LLM decides whether to continue or finishmessages = [system_prompt, user_request]
while True:
response = llm.invoke(messages, tools=tool_definitions)
if response.is_final_answer:
print(response.content)
break
tool_name = response.tool_call.name
tool_args = response.tool_call.args
result = execute_tool(tool_name, tool_args)
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "tool", "content": str(result)})Every tool needs four things:
tools = [{
"name": "search_web",
"description": "Search the web for current information",
"input_schema": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "The search query"},
"max_results": {"type": "integer", "default": 5}
},
"required": ["query"]
}
}]Every tool call should include a reasoning field. This forces the LLM to explain WHY before it acts:
def search_web(query: str, reasoning: str, max_results: int = 5):
"""Search the web. The 'reasoning' param explains why this call is needed."""
...This single pattern:
| Mode | Behavior | Use When |
|---|---|---|
auto | LLM decides to call tool or respond | Simple tasks, human handoff |
any / required | LLM MUST call a tool | Agent loops where each turn = action |
| Specific tool | LLM must use exact tool | Debugging, testing, routing |
Risk compounds with runtime — A 1% per-turn failure rate = 63% chance of disaster over 100 turns. Long-running agents aren't safer, they're more exposed.
If your agent can write AND execute code, you're back at L1 security — The marquee break: agent writes cleanup.py, runs python cleanup.py, your production data is gone.
Every turn is a roll of the dice — Modern models refuse well 99% of the time. That last 1% grows with every feature release. Engineer the harness for the 1%, not the 99%.
Token costs scale with loop depth, not task complexity — A simple task with a bad loop costs 10x more than a complex task with a clean loop. Optimize the loop first.
What you can't measure, you can't improve — Every agent system needs: cost per task, success rate, loop efficiency, failure mode tracking.
Objective: Build a single-tool agent from scratch in under 50 lines.
Starter code: course/labs/L1-first-agent/starter.py
# TODO: Fill in the agent loop
# 1. Define a tool (search_web or file_read)
# 2. Define the tool schema
# 3. Implement the agent loop
# 4. Handle tool results
# 5. Print the final answer
import json
# Your code here...Checkpoints:
Solution: course/labs/L1-first-agent/solution.py
reasoning parameter do?