240 lines
8.1 KiB
Markdown
240 lines
8.1 KiB
Markdown
# Agentic Engineering — Technical Field Manual
|
||
|
||
A consolidated quick reference for all course concepts, commands, and patterns.
|
||
|
||
---
|
||
|
||
## 1. Agent CLI Quick Reference
|
||
|
||
### Claude Code
|
||
```bash
|
||
claude --init # Project setup
|
||
claude --teammate-mode tmux # Multi-agent teams (split panes)
|
||
claude -p "prompt" --print # Headless execution
|
||
claude --allowedTools "Read Write" # Constrain tool surface
|
||
claude --hooks .claude/hooks/ # Custom hooks directory
|
||
```
|
||
|
||
### Pi Coding Agent
|
||
```bash
|
||
pi -e extensions/damage-control.ts # Security auditing
|
||
pi -e extensions/tilldone.ts # Task discipline
|
||
pi -e extensions/coms.ts # P2P agent communication
|
||
pi --mode rpc # Programmatic control (26+ commands)
|
||
pi -e extensions/agent-team.ts # Dispatcher orchestration
|
||
pi -e extensions/agent-chain.ts # Pipeline orchestration
|
||
```
|
||
|
||
### OpenCode
|
||
```bash
|
||
opencode run "task" # Headless execution
|
||
opencode --model opencode-go/deepseek-v4-flash run # Specific model
|
||
opencode --config ~/.grok/config.toml run # Custom config
|
||
```
|
||
|
||
### OpenClaw
|
||
```bash
|
||
claw start --daemon # Always-on employee
|
||
claw do "task description" # One-shot execution
|
||
claw schedule --cron "0 6 * * 1" --task # Recurring task
|
||
```
|
||
|
||
### Install Course
|
||
```bash
|
||
# Windows
|
||
powershell -File install.ps1
|
||
|
||
# Linux/Mac
|
||
bash install.sh
|
||
bash install.sh security # Single kit
|
||
```
|
||
|
||
---
|
||
|
||
## 2. The 6-Level Security Ladder
|
||
|
||
```
|
||
L0: ACIP (prompt injection defense) — Costs nothing, blocks simple attacks
|
||
L1: safe-mode skill — Theatre (model can override)
|
||
L2: --append-system-prompt — Theatre+ (model can still override)
|
||
L3: Bash blacklist hook — Reactive (use as FLOOR)
|
||
L4: Bash whitelist hook — Architectural (only N safelisted cmds)
|
||
L5: No bash, custom tools only — Production-grade (bash doesn't exist)
|
||
```
|
||
|
||
### Probability Math
|
||
```python
|
||
P_failure = 1 - (1 - p)^N
|
||
# p=0.01 (1%), N=100: 63.4% chance of disaster
|
||
# p=0.001 (0.1%), N=100: 9.5%
|
||
# p=0.0001 (0.01%), N=1000: 9.5%
|
||
```
|
||
|
||
### Damage Control — 3 Access Levels
|
||
```yaml
|
||
zeroAccessPaths: [".env", "~/.ssh/", "*.pem"] # Can't read/write
|
||
readOnlyPaths: ["package-lock.json", "node_modules/"] # Can read only
|
||
noDeletePaths: [".git/", "Dockerfile"] # Can modify, can't delete
|
||
```
|
||
|
||
---
|
||
|
||
## 3. Orchestration Patterns
|
||
|
||
| Pattern | Structure | Best For | Tool |
|
||
|---------|-----------|----------|------|
|
||
| **P-Thread** | N agents parallel, pick best | Creative work, benchmarking | mprocs |
|
||
| **F-Thread** | N agents → Judge → Winner | Code gen, UI gen | agent-team |
|
||
| **B-Thread** | Orchestrator → Leads → Workers | Production systems | lead-agents |
|
||
| **C-Thread** | Agent → Checkpoint → Human | High-stakes work | tilldone |
|
||
| **L-Thread** | Long-running hours+ | Background tasks | OpenClaw daemon |
|
||
| **Agent Chain** | Step1 → Step2 → Step3 | Plan→Build→Review | agent-chain |
|
||
|
||
### YAML Pipeline Template
|
||
```yaml
|
||
steps:
|
||
- agent: planner
|
||
prompt: "Create plan for: $INPUT"
|
||
- agent: builder
|
||
prompt: "Implement: $INPUT"
|
||
- agent: reviewer
|
||
prompt: "Review: $INPUT"
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Key Formulas
|
||
|
||
### Compute Advantage
|
||
```
|
||
CA = (Compute Scaling × Autonomy) ÷ (Time + Effort + Monetary Cost)
|
||
```
|
||
|
||
### pass@k
|
||
```
|
||
pass@k = 1 - (1 - p)^k
|
||
# p=0.6 (60% single), k=3: 93.6%
|
||
# p=0.6, k=5: 98.9%
|
||
```
|
||
|
||
### 3x Cost Rule
|
||
```
|
||
Production cost = Prototype cost × 3.0
|
||
With retries = Prototype cost × 1.5
|
||
```
|
||
|
||
### Cascade Routing Savings
|
||
```
|
||
Savings = 1 - (cascade_cost / single_model_cost)
|
||
# Typical: 66-80% savings
|
||
```
|
||
|
||
---
|
||
|
||
## 5. Model Pricing (per million tokens)
|
||
|
||
| Model | Input | Output | Best For |
|
||
|-------|-------|--------|----------|
|
||
| Gemini 2.5 Flash | $0.15 | $0.60 | High-volume, simple |
|
||
| DeepSeek V3 | $0.27 | $1.10 | Structured tasks |
|
||
| Claude Sonnet 4 | $3.00 | $15.00 | General agentic |
|
||
| Claude Opus 4 | $15.00 | $75.00 | Complex planning |
|
||
| GPT-5 | $10.00 | $40.00 | Frontier reasoning |
|
||
|
||
### Cascade Routing Strategy
|
||
```
|
||
Simple retrieval → Gemini Flash ($0.15/$0.60)
|
||
Analysis → Claude Sonnet ($3/$15)
|
||
Critical decision → Claude Opus ($15/$75)
|
||
Formatting → Gemini Flash ($0.15/$0.60)
|
||
```
|
||
|
||
---
|
||
|
||
## 6. Verifier Confidence Ladder
|
||
|
||
| Level | Meaning | Bar Color |
|
||
|-------|---------|-----------|
|
||
| PERFECT | Every claim verified, zero gaps | Green |
|
||
| VERIFIED | All passed, minor non-blocking gaps | Green |
|
||
| PARTIAL | No failures, significant unverifiable gaps | Orange |
|
||
| FEEDBACK | At least one claim failed, correction sent | Orange |
|
||
| FAILED | Cannot verify, escalating to human | Red |
|
||
|
||
---
|
||
|
||
## 7. Lab Index
|
||
|
||
| Lab | Module | Topic | Files |
|
||
|-----|--------|-------|-------|
|
||
| L1 | M1 | First Agent (single-tool) | starter.py, solution.py |
|
||
| L2a | M2 | Multi-Tool Agent | starter.py, solution.py |
|
||
| L2b | M2 | Context-Aware Agent | starter.py, solution.py |
|
||
| L3a | M3 | Whitelist Hook (L4) | starter.py, solution.py |
|
||
| L3b | M3 | Verifier Agent | starter.py, solution.py |
|
||
| L4a | M4 | Agent Chain (YAML) | starter.yaml, solution.yaml |
|
||
| L4b | M4 | Multi-Team Config | starter.yaml, solution.yaml |
|
||
| L5a | M5 | Observability (SQLite) | starter.py, solution.py |
|
||
| L5b | M5 | CI/CD Pipeline | starter.py, solution.py |
|
||
| L6a | M6 | Eval Harness (pass@k) | starter.py, solution.py |
|
||
| L6b | M6 | Cost Optimization | starter.py, solution.py |
|
||
| L7a | M7 | Autoresearch Loop | starter.py, solution.py |
|
||
| L7b | M7 | Meta-Agent | starter.py, solution.py |
|
||
|
||
Run offline: `python -c "from mock_llm import MockAnthropic; print('Mock LLM ready')"`
|
||
|
||
---
|
||
|
||
## 8. Capstone Options
|
||
|
||
| Project | Difficulty | Description | Reference |
|
||
|---------|-----------|-------------|-----------|
|
||
| Brand Monitor | Intermediate | Multi-LLM brand scanning | capstone-reference/brand-monitor/ |
|
||
| Code Review Pipeline | Int-Adv | Plan→Build→Review→Verify chain | — |
|
||
| Strategic Decision Board | Advanced | 8-agent CEO board | ceo-agents/ |
|
||
|
||
---
|
||
|
||
## 9. Skill Kits (7 kits, 18 SKILL.md files)
|
||
|
||
| Kit | Price | Skills | Install |
|
||
|-----|-------|--------|---------|
|
||
| Security Foundation | $49 | L3, L4, L5, damage-control, sandbox | `bash install.sh security` |
|
||
| Multi-Agent Orchestration | $49 | teams, chains, mental-model, domain-lock | `bash install.sh orchestration` |
|
||
| Verifier Pro | $39 | builder, confidence, decomposition | `bash install.sh verifier` |
|
||
| Task Discipline | $29 | tilldone core, progress, nudge, purpose | `bash install.sh task` |
|
||
| Autoresearch | $39 | experiment loop, integrity, median | `bash install.sh research` |
|
||
| Observability | $29 | tracer, cost, replay, loop-detect | `bash install.sh observability` |
|
||
| CEO Board | $49 | 11 agents + verifier + tracker | `bash install.sh board` |
|
||
| **Enterprise** | **$199** | **All 7 kits** | **`bash install.sh`** |
|
||
|
||
---
|
||
|
||
## 10. The Universal Truth
|
||
|
||
> **Deterministic orchestrates non-deterministic. Code is the harness. AI is the engine. Together they're unstoppable.**
|
||
|
||
| Creator | Their Version |
|
||
|---------|---------------|
|
||
| Kelsey Hightower | "Zero Token Architecture — don't waste tokens on deterministic work" |
|
||
| Mario Zechner | "Write architecture by hand. Let agents do the boring stuff" |
|
||
| Daniel Miessler | "Code Before Prompts. If bash can do it, don't use AI" |
|
||
| IndyDevDan | "ADW — deterministic code orchestrates non-deterministic agents" |
|
||
| Anders Hejlsberg | "Don't ask AI for the answer. Ask it to write a program" |
|
||
| Armin Ronacher | "Pi's harness layer is worth maintaining carefully because it solves hard problems" |
|
||
|
||
---
|
||
|
||
## 11. Quick Reference Card
|
||
|
||
```
|
||
AGENT LOOP: Think → Act → Observe → Repeat
|
||
HARNESS: Instructions + Tools + Environment + State + Verification
|
||
CONTROL: Context + Model + Prompt + Tools
|
||
SECURITY: L0(ACIP) → L1(skill) → L2(prompt) → L3(blacklist) → L4(whitelist) → L5(no-bash)
|
||
ORCHESTRATION: Dispatcher | Pipeline | P2P | P-Thread | F-Thread | B-Thread
|
||
ECONOMICS: CA = (CS × A) ÷ (T + E + MC) 3x rule Cascade 66-80%
|
||
EVALS: pass@k = 1 - (1-p)^k Cost/task Tool call accuracy
|
||
TRUST: "Yes, because I've engineered it" — not blind faith
|
||
```
|