112 lines
5.8 KiB
Markdown
112 lines
5.8 KiB
Markdown
# The Great Agent Debate
|
|
|
|
Two opposing views from credible engineers. Both are essential context for any agentic engineering course.
|
|
|
|
---
|
|
|
|
## The Optimist: Armin Ronacher — "Building Pi With Pi"
|
|
|
|
**Who**: Creator of Flask, Jinja2, Click, Sentry co-founder. Maintainer of Pi Coding Agent.
|
|
**Post**: https://lucumr.pocoo.org/2026/5/24/pi-oss/
|
|
**Date**: May 24, 2026
|
|
|
|
### Key Arguments
|
|
|
|
1. **Pi is built with Pi** — The core thesis: agents are now capable enough to help build the tools that build agents. This is the meta-agent pattern (M7).
|
|
|
|
2. **The harness matters most** — "Pi's harness layer is worth maintaining carefully because it solves hard problems." This validates M1's central thesis.
|
|
|
|
3. **We're not at full autonomy yet** — "We're quite far off today from where Bun and [others] are. Today it does not seem like we know how to pull off a dark factory." Honest admission of current limitations.
|
|
|
|
4. **Open source challenges** — AI-generated PRs and automated issue closers create new maintenance burdens. GitHub isn't built for AI-native OSS workflows.
|
|
|
|
### Course Relevance
|
|
|
|
| Argument | Course Module |
|
|
|----------|---------------|
|
|
| "Built with Pi" — agents building agents | M7 Meta-Agents |
|
|
| Harness layer is the product | M1 Harness vs Model |
|
|
| Not at full autonomy yet | M1 Vibe Coding vs Engineering |
|
|
| OSS maintenance challenges | M5 Production Patterns |
|
|
|
|
---
|
|
|
|
## The Pessimist: George Hotz — "The Eternal Sloptember"
|
|
|
|
**Who**: Founder of comma.ai. First iPhone jailbreaker. Winner of DEF CON CTF.
|
|
**Post**: https://geohot.github.io//blog/jekyll/update/2026/05/24/the-eternal-sloptember.html
|
|
**Date**: May 24, 2026
|
|
|
|
### Key Arguments
|
|
|
|
1. **"Agents cannot program"** — They produce output that looks correct but is broken in ways that are getting harder to detect. Statistical mimicry, not understanding.
|
|
|
|
2. **"It's not 'you're using it wrong'"** — He's tried all models, all harnesses, all prompts. The conclusion is the same. The slot machine analogy: "frontloads all the progress, then gives you a slot machine lever to pull to hope it gets the polish done. It never quite gets there."
|
|
|
|
3. **AI ≠ software engineer** — Useful as a better Google, great for prototypes. But "not close to the bar at any company I have worked at."
|
|
|
|
4. **Agents hurt large orgs more** — Small teams can error-correct. Large orgs have slow feedback loops. Bottom performers produce 10x slop. "Golden era for buckets of slop, dark age for gems of quality."
|
|
|
|
5. **Will macOS get better or worse?** — Concrete challenge: Apple is pushing AI on all engineers. Will the OS improve or degrade?
|
|
|
|
6. **World models needed** — "Real programming agents will need world models, not some RLVR shit that comments out the failing test."
|
|
|
|
### Course Relevance
|
|
|
|
| Argument | Course Module |
|
|
|----------|---------------|
|
|
| Agents can't program | M1 Vibe Coding vs Engineering (counterpoint) |
|
|
| Not "using it wrong" | M1 Decision Framework |
|
|
| Slop amplification at scale | M5 Quality Gates, M6 Evals |
|
|
| World models needed | M7 Future Directions |
|
|
| Slot machine polish problem | M3 Verifier Pattern (the solution) |
|
|
|
|
---
|
|
|
|
## Course Integration
|
|
|
|
### M1 Foundations — Add "The Counterargument"
|
|
|
|
After the "Do You Trust Your Agents?" lesson, add a new section:
|
|
|
|
> Not everyone agrees. George Hotz, founder of comma.ai and one of the most respected engineers in the industry, argues that AI agents "cannot program" and their adoption "will be one of the most costly mistakes in the field's history."
|
|
>
|
|
> His core argument: agents produce statistically plausible output that is broken in increasingly hard-to-detect ways. The "slot machine" problem — agents frontload progress quickly, then require endless tweaking to finish.
|
|
>
|
|
> Why we include this: If you only hear the optimistic view, you'll deploy agents naively and get burned. Hotz's critique isn't wrong — it identifies real failure modes that our course addresses:
|
|
> - **Slop detection** → M3 Verifier Pattern
|
|
> - **Quality gates** → M5 CI/CD for Agents
|
|
> - **Evaluation** → M6 Golden Datasets
|
|
> - **Trust but verify** → M3 Defense-in-Depth
|
|
|
|
### M4 Orchestration — Add Armin's "Built With Pi"
|
|
|
|
After the Meta-Agents lesson:
|
|
|
|
> Armin Ronacher, creator of Flask and Pi maintainer, revealed that Pi itself is built using Pi agents. This is the meta-agent pattern in production: the tool that builds agents is itself built by agents.
|
|
>
|
|
> However, he's honest about limitations: "We're quite far off today from where Bun and [others] are. Today it does not seem like we know how to pull off a dark factory."
|
|
>
|
|
> This is the honest middle ground between full automation and pure skepticism.
|
|
|
|
### M5 Production — Add George's Quality Warning
|
|
|
|
After the CI/CD lesson:
|
|
|
|
> George Hotz predicts agents will "end up hurting large organizations more than high performing individuals or small orgs." His reasoning: large orgs have slow feedback loops, bottom performers can't error-correct, and agent-amplified slop will degrade quality at scale.
|
|
>
|
|
> This is why M5 exists. The production patterns in this module (CI/CD gates, golden datasets, shadow deployments, observability, rollback) are the organizational immune system against agent-amplified slop. Without them, Hotz is right.
|
|
|
|
---
|
|
|
|
## The Synthesized View
|
|
|
|
| Armin Says | George Says | What We Teach |
|
|
|-----------|-------------|---------------|
|
|
| The harness is the product | The output is slop | The harness catches the slop (M3) |
|
|
| Not at dark factory yet | Agents can't program | Use agents for what they're good at (M1) |
|
|
| Built with Pi is real | Slot machine polish problem | Verifier pattern finishes the polish (M3) |
|
|
| OSS maintenance is hard | Large orgs will suffer | Production patterns prevent this (M5) |
|
|
|
|
**The honest take**: Both are right. Agents produce slop. The harness, verification, and production patterns are what turn slop into shipped quality. The course teaches the harness. Without the harness, Hotz wins.
|