# Context Window Management for AI Agents

**June 18, 2026**

Your agent's context window is its working memory. Fill it with the wrong things, and the agent makes bad decisions. Fill it with too much, and you're burning $0.15 per call on irrelevant tokens.

---

## The Context Budget

Every token in the context window has a cost — literally (API pricing) and figuratively (attention dilution). The key insight: **not all tokens are equal**.

```
High-Value Tokens (always include):
├── System prompt (persona, rules, constraints)
├── Tool definitions (name, description, schema)
├── Current user request
└── Most recent tool results

Medium-Value Tokens (include if relevant):
├── Conversation history (last 3-5 exchanges)
├── Mental model files (agent-specific expertise)
└── Reference documents (API docs, style guides)

Low-Value Tokens (exclude in production):
├── Entire conversation history (use summaries instead)
├── Large reference files (link instead of inline)
├── Previous tool results that are no longer relevant
└── System messages older than the last 10 turns
```

---

## Sliding Window Strategy

The most common production approach. Keep the window focused on recent + important:

```python
class SlidingWindow:
    def __init__(self, max_tokens=32000):
        self.max_tokens = max_tokens
        self.system_prompt = ""  # always retained
        self.messages = []       # conversation history
        self.token_count = 0

    def add_message(self, msg):
        self.messages.append(msg)
        self.token_count += count_tokens(msg)

        # Trim to fit budget — remove oldest tool results first
        while self.token_count > self.max_tokens and len(self.messages) > 3:
            removed = self.messages.pop(1)  # keep system + last user
            self.token_count -= count_tokens(removed)
```

**Best for**: Chat-style agents, interactive coding assistants, support bots.

**Limitation**: Loses context from early in the conversation. If the user mentions something important 20 turns ago, the agent won't remember it.

---

## Summarization Strategy

Periodically summarize the conversation into condensed context:

```python
def summarize_context(messages):
    prompt = f"Summarize this conversation in 3-5 sentences, \
               preserving key decisions and user preferences: \
               {messages[-20:]}"
    summary = llm.call(prompt)
    return {"role": "system", "content": f"[CONTEXT: {summary}]"}
```

**Pattern**:
```
Turns 1-10: full messages
Turn 11: summarize turns 1-10 → insert as system message
Turns 11-20: full messages (with summary in system prompt)
Turn 21: summarize turns 11-20 → update system summary
... repeat
```

**Best for**: Long-running research agents, complex multi-step tasks, customer support.

**Cost**: Each summarization costs ~50-100 tokens. That's $0.0003-0.0015 per summary — essentially free.

---

## Mental Model Strategy (Advanced)

Give the agent its own persistent memory that it reads and writes:

```python
# Agent reads this file at the start of every session
MENTAL_MODEL = "agent-expertise.md"

def load_mental_model():
    if os.path.exists(MENTAL_MODEL):
        return open(MENTAL_MODEL).read()
    return ""

def save_observation(key, value):
    with open(MENTAL_MODEL, "a") as f:
        f.write(f"\n- {key}: {value}")
```

The agent builds expertise over time. This is the most token-efficient strategy because the agent curates what it remembers — it doesn't keep everything.

**Best for**: Specialized agents (code reviewers, security auditors, data analysts) that work on multiple disjoint tasks.

---

## Token Budget Allocation Formula

```
Budget = SystemPrompt(15%) + Tools(15%) + Conversation(40%) + ToolResults(30%)

If budget is tight:
  1. Shorten tool descriptions (remove examples)
  2. Summarize conversation history (keep last 3-5 full, summarize the rest)
  3. Trim tool results to only the relevant sections
  4. Remind the agent about mental models instead of re-reading them
```

---

## The 80/20 Rule of Context Management

80% of context problems come from 20% of the causes:

1. **Too much conversation history** — keep last 5 exchanges, summarize the rest
2. **Too many tool results** — only keep results that were directly used
3. **Redundant system instructions** — don't repeat the same rules in every system message
4. **Large reference files** — cite them, don't include them inline

Fix these four, and you solve most context window issues.

---

*This is adapted from Module 2: Architecture of the [Agentic Engineering the Hard Way](/free-preview) course. Full course includes 65 lessons, 13 labs, and 20 skill kits.*