134 lines
4.6 KiB
Markdown
134 lines
4.6 KiB
Markdown
# How to Choose the Right Model for Your Agent
|
|
|
|
**June 22, 2026**
|
|
|
|
You're building an agent. Which model do you use? The answer isn't "the best one" — it's "the right one for this subtask."
|
|
|
|
---
|
|
|
|
## The Model Landscape (Mid-2026)
|
|
|
|
| Tier | Models | Cost/1M tokens | Best For |
|
|
|------|--------|----------------|----------|
|
|
| **Frontier** | Claude Opus 4, GPT-5, Gemini Ultra 2.0 | $12-30 | Complex reasoning, code generation, safety-critical decisions |
|
|
| **Mid** | Claude Sonnet 4, GPT-5-mini, Gemini Pro 2.0 | $3-8 | Analysis, classification, summarization |
|
|
| **Cheap** | Claude Haiku 3.5, GPT-5-flash, Gemini Flash 2.0 | $0.15-0.50 | Bulk processing, extraction, simple tool calls |
|
|
| **Local** | Llama 4, DeepSeek Coder, Mistral Large 2 | Free (HW cost) | Private data, offline, latency-sensitive |
|
|
|
|
The price range from cheapest to most expensive is **200x**. Using the wrong tier for a task is like renting a dump truck to move a shoebox.
|
|
|
|
---
|
|
|
|
## The Decision Framework
|
|
|
|
```
|
|
Task complexity
|
|
│
|
|
├─ Is it deterministic? (grep, parse, format, extract)
|
|
│ → Use CHEAP model (Haiku/Flash)
|
|
│
|
|
├─ Does it need reasoning? (analyze, explain, plan)
|
|
│ → Use MID model (Sonnet/Pro)
|
|
│
|
|
├─ Is it customer-facing? (report, email, summary)
|
|
│ → Use FRONTIER model (Opus/GPT-5)
|
|
│
|
|
└─ Does it involve private data? (PII, IP, secrets)
|
|
→ Use LOCAL model (Llama/DeepSeek)
|
|
```
|
|
|
|
---
|
|
|
|
## Cascade Routing in Practice
|
|
|
|
```python
|
|
MODEL_TIERS = {
|
|
"cheap": {
|
|
"model": "claude-haiku-3.5-20260501",
|
|
"cost_per_m_tokens": 0.15,
|
|
"max_tokens": 4000,
|
|
},
|
|
"mid": {
|
|
"model": "claude-sonnet-4-20260501",
|
|
"cost_per_m_tokens": 3.00,
|
|
"max_tokens": 8000,
|
|
},
|
|
"premium": {
|
|
"model": "claude-opus-4-20260501",
|
|
"cost_per_m_tokens": 15.00,
|
|
"max_tokens": 16000,
|
|
},
|
|
}
|
|
|
|
def route_to_tier(task_type, prompt):
|
|
if task_type in ("extract", "parse", "format", "search", "classify"):
|
|
tier = "cheap"
|
|
elif task_type in ("analyze", "explain", "plan", "review"):
|
|
tier = "mid"
|
|
elif task_type in ("generate", "synthesize", "report", "decide"):
|
|
tier = "premium"
|
|
else:
|
|
tier = "mid" # safe default
|
|
|
|
model = MODEL_TIERS[tier]
|
|
response = call_llm(model["model"], prompt, model["max_tokens"])
|
|
cost = (count_tokens(prompt) / 1_000_000) * model["cost_per_m_tokens"]
|
|
|
|
return {"response": response, "tier": tier, "cost": cost}
|
|
```
|
|
|
|
---
|
|
|
|
## Model Selection Anti-Patterns
|
|
|
|
### "Just Use the Best Model"
|
|
|
|
The most expensive anti-pattern. Running every task through Opus costs 100x more than routing simple tasks to Haiku. For a production agent making 500 calls/day:
|
|
- All Opus: ~$75/day
|
|
- Cascaded: ~$8/day
|
|
|
|
**Savings: 89%**
|
|
|
|
### "Use the Cheapest Model Everywhere"
|
|
|
|
Saves money, loses quality. Cheap models hallucinate more, follow instructions less reliably, and produce worse output on complex tasks. A single bad output from a cheap model can cost more in debugging time than you saved in API fees.
|
|
|
|
### "One Model Per Agent"
|
|
|
|
This is acceptable for simple agents but misses optimization opportunities. Within a single agent session, you can route different subtasks to different models. The same agent can use Haiku for file discovery and Opus for synthesis.
|
|
|
|
---
|
|
|
|
## When Local Models Make Sense
|
|
|
|
Local models (Llama 4, DeepSeek) are not competitive with cloud APIs on quality. But they win on:
|
|
|
|
1. **Privacy** — Data never leaves your machine
|
|
2. **Latency** — No network calls (5ms vs 500ms)
|
|
3. **Cost at scale** — Free after hardware purchase
|
|
4. **Offline operation** — Works without internet
|
|
|
|
**Best use cases**: Code completion, local file analysis, private document review, development assistance.
|
|
|
|
**Worst use cases**: Complex reasoning, multi-step planning, tasks requiring up-to-date knowledge.
|
|
|
|
---
|
|
|
|
## The 80/20 Rule of Model Selection
|
|
|
|
80% of cost savings come from one change: **stop using frontier models for routine work**.
|
|
|
|
| Task Type | Current Model | Recommended Model | Savings |
|
|
|-----------|--------------|-------------------|---------|
|
|
| File discovery | Opus/Sonnet | Haiku/Flash | 95% |
|
|
| Data extraction | Opus/Sonnet | Haiku/Flash | 95% |
|
|
| Classification | Opus/Sonnet | Haiku/Flash | 95% |
|
|
| Analysis | Opus | Sonnet | 80% |
|
|
| Code review | Opus | Sonnet | 80% |
|
|
| Report generation | Opus | Opus (keep) | 0% |
|
|
| Complex reasoning | Opus | Opus (keep) | 0% |
|
|
|
|
---
|
|
|
|
*This is adapted from Module 6: Economics of the [Agentic Engineering the Hard Way](/free-preview) course. Full course includes 65 lessons, 13 labs, and 20 skill kits.*
|