4.6 KiB

Raw Permalink Blame History

How to Choose the Right Model for Your Agent

June 22, 2026

You're building an agent. Which model do you use? The answer isn't "the best one" — it's "the right one for this subtask."

The Model Landscape (Mid-2026)

Tier	Models	Cost/1M tokens	Best For
Frontier	Claude Opus 4, GPT-5, Gemini Ultra 2.0	$12-30	Complex reasoning, code generation, safety-critical decisions
Mid	Claude Sonnet 4, GPT-5-mini, Gemini Pro 2.0	$3-8	Analysis, classification, summarization
Cheap	Claude Haiku 3.5, GPT-5-flash, Gemini Flash 2.0	$0.15-0.50	Bulk processing, extraction, simple tool calls
Local	Llama 4, DeepSeek Coder, Mistral Large 2	Free (HW cost)	Private data, offline, latency-sensitive

The price range from cheapest to most expensive is 200x. Using the wrong tier for a task is like renting a dump truck to move a shoebox.

The Decision Framework

Task complexity
      │
      ├─ Is it deterministic? (grep, parse, format, extract)
      │     → Use CHEAP model (Haiku/Flash)
      │
      ├─ Does it need reasoning? (analyze, explain, plan)
      │     → Use MID model (Sonnet/Pro)
      │
      ├─ Is it customer-facing? (report, email, summary)
      │     → Use FRONTIER model (Opus/GPT-5)
      │
      └─ Does it involve private data? (PII, IP, secrets)
            → Use LOCAL model (Llama/DeepSeek)

Cascade Routing in Practice

MODEL_TIERS = {
    "cheap": {
        "model": "claude-haiku-3.5-20260501",
        "cost_per_m_tokens": 0.15,
        "max_tokens": 4000,
    },
    "mid": {
        "model": "claude-sonnet-4-20260501",
        "cost_per_m_tokens": 3.00,
        "max_tokens": 8000,
    },
    "premium": {
        "model": "claude-opus-4-20260501",
        "cost_per_m_tokens": 15.00,
        "max_tokens": 16000,
    },
}

def route_to_tier(task_type, prompt):
    if task_type in ("extract", "parse", "format", "search", "classify"):
        tier = "cheap"
    elif task_type in ("analyze", "explain", "plan", "review"):
        tier = "mid"
    elif task_type in ("generate", "synthesize", "report", "decide"):
        tier = "premium"
    else:
        tier = "mid"  # safe default

    model = MODEL_TIERS[tier]
    response = call_llm(model["model"], prompt, model["max_tokens"])
    cost = (count_tokens(prompt) / 1_000_000) * model["cost_per_m_tokens"]

    return {"response": response, "tier": tier, "cost": cost}

Model Selection Anti-Patterns

"Just Use the Best Model"

The most expensive anti-pattern. Running every task through Opus costs 100x more than routing simple tasks to Haiku. For a production agent making 500 calls/day:

All Opus: ~$75/day
Cascaded: ~$8/day

Savings: 89%

"Use the Cheapest Model Everywhere"

Saves money, loses quality. Cheap models hallucinate more, follow instructions less reliably, and produce worse output on complex tasks. A single bad output from a cheap model can cost more in debugging time than you saved in API fees.

"One Model Per Agent"

This is acceptable for simple agents but misses optimization opportunities. Within a single agent session, you can route different subtasks to different models. The same agent can use Haiku for file discovery and Opus for synthesis.

When Local Models Make Sense

Local models (Llama 4, DeepSeek) are not competitive with cloud APIs on quality. But they win on:

Privacy — Data never leaves your machine
Latency — No network calls (5ms vs 500ms)
Cost at scale — Free after hardware purchase
Offline operation — Works without internet

Best use cases: Code completion, local file analysis, private document review, development assistance.

Worst use cases: Complex reasoning, multi-step planning, tasks requiring up-to-date knowledge.

The 80/20 Rule of Model Selection

80% of cost savings come from one change: stop using frontier models for routine work.

Task Type	Current Model	Recommended Model	Savings
File discovery	Opus/Sonnet	Haiku/Flash	95%
Data extraction	Opus/Sonnet	Haiku/Flash	95%
Classification	Opus/Sonnet	Haiku/Flash	95%
Analysis	Opus	Sonnet	80%
Code review	Opus	Sonnet	80%
Report generation	Opus	Opus (keep)	0%
Complex reasoning	Opus	Opus (keep)	0%

This is adapted from Module 6: Economics of the Agentic Engineering the Hard Way course. Full course includes 65 lessons, 13 labs, and 20 skill kits.

4.6 KiB Raw Permalink Blame History