Skip to content

Context Management

If you learn one thing from this knowledge base, learn this. Context management is the single highest-leverage skill in working with AI, and it is the biggest gap between someone who tried an AI tool once and someone who gets compounding value from it every day. Most "the AI is dumb" moments are actually context problems.

Why this matters — Limitation #1 (Limited memory). A model has a fixed context window. Everything it "knows" in a given moment — your instructions, the files it has read, the conversation so far — competes for that finite space. Worse, models don't use a full context evenly: they pay the most attention to the beginning and end and can lose track of things buried in the middle (the "lost in the middle" problem). As the context fills with tool output, dead ends, and tangents, quality quietly degrades. Managing context is managing the model's effective intelligence.

The mental model: context is working memory, not storage

Think of the context window as the model's desk, not its filing cabinet. A desk has limited space. If you pile every document you own onto it, the model can't find the one it needs — and the older items slide off the edge. Your job is to keep the right things on the desk and the rest in the cabinet until needed.

This reframes the whole skill. You are not trying to give the model everything. You are curating exactly what it needs right now.

Point the agent at the right files

When an agent doesn't know where something lives, it scans — opening files, searching, reading directories. Every one of those reads lands in the context window, much of it irrelevant. By the time it finds the right file, the desk is cluttered with ten wrong ones.

Instead, point it directly:

  • "The bug is in services/payments/refund.ts. Read that and its test file, then propose a fix."
  • "The retry logic lives in core/queue/. Don't look anywhere else."

You almost always know roughly where the relevant code is. Spending that knowledge up front saves a dozen wasted reads and keeps the context clean. When you genuinely don't know, ask the agent to find it first and report back the file path — then start fresh and point at it.

Clear and refresh when the context gets noisy

Long sessions accumulate cruft: abandoned approaches, large command outputs, a debugging rabbit hole you backed out of. The model is still "looking at" all of it, and it drags quality down.

When you notice the agent getting confused, repeating itself, or referencing things you've moved past, clear the context and restart with a clean summary. A fresh session that begins with "Here's where we are and here's the next step" almost always outperforms a bloated one that's been running for an hour.

Tell-tale signs of context degradation: the agent re-suggests something you already rejected, "forgets" a constraint you set earlier, mixes up two files, gets slower and more hedged, or confidently contradicts itself. These are signals to refresh, not to argue. For a fuller catalogue of these cues and how to act on them, see Knowing When to Reset.

The highest-value pattern: investigate → summarize → execute fresh

This is the technique that separates experts. It has three moves:

  1. Investigate in one session. Let the agent explore freely — read code, trace the data flow, search, hit dead ends. This session is meant to get messy; exploration is noisy by nature.
  2. Summarize the findings into a markdown file. Ask the agent to write down what it learned: the relevant files, the root cause, the constraints, the proposed approach. This distills a sprawling, polluted context into a few clean paragraphs.
  3. Start a FRESH session to execute. Open a new session with only that markdown summary plus the specific files needed. The model now works from a clean desk containing exactly the distilled signal — no exploration debris.

The payoff: the execution session spends its entire context budget on the actual work, not on the mess that exploration created. You get the benefit of deep investigation without paying its context cost during execution.

This pattern also produces a durable artifact — the markdown summary survives the session and helps the next person (or the next you). It connects directly to Plan and Design First, where the written plan plays the same role.

Markdown files as external memory

That summary doesn't have to be a one-off. A plain .md file in the repo is the most underrated context tool you have: cheap to create, readable by both you and the agent, and — crucially — it lives outside the context window, so it survives resets, compaction, and closing the session entirely. Lean on it deliberately.

A few shapes worth abusing:

  • A scratchpad for a messy investigation — "write what you find to notes.md as you go." The exploration debris lands in the file instead of clogging the conversation, and you keep the distilled trail.
  • A progress log / running TODO — an ordered checklist the agent ticks off as it works. Because it's on disk, it is your state: stop mid-task, open a fresh session tomorrow, point at the file, and resume with nothing lost. This is exactly what a plan file does — the same trick, generalized.
  • A handoff note — a short "where things stand, what's left, what not to touch" for the next session, the next teammate, or future-you. See From Feature to Plan to Handoff.
  • A decisions file — the why behind choices the code doesn't explain, so a later session doesn't relitigate them.

The move in all four is the same: push state out of the fragile conversation and onto durable disk, then pull back only the slice you need. The chat is working memory; the file is the record.

Keep these separate from committed project memory. These are throwaway working files — different from CLAUDE.md and repo rules, which are curated, auto-loaded, and shared with the team. Keep transient scratch and progress notes in git-ignored or docs/plans/-style paths so they don't pollute the durable memory everyone loads. See Memory and Project Rules.

Compaction: the automatic version (and its limits)

Most agent tools now do a version of the investigate → summarize move automatically, called compaction. When a session approaches the context limit, the tool condenses everything so far into a shorter summary and continues from that, discarding the verbose original — the tool reclaiming desk space on your behalf so the session can keep going.

Compaction is genuinely useful, but understand its limit: a machine summary is lossy. It may quietly drop the one constraint or file path you cared about, and you don't choose what survives. So:

  • For anything important, prefer a deliberate, human-checked summary (the pattern above) over letting auto-compaction decide what to keep.
  • Treat a compaction event as a cue to sanity-check that the agent still remembers the key facts — restate them if not.

This is the deeper reason a focused context beats a giant one: a small, curated context never needs compressing, so nothing gets lost and every token is signal. A bloated context the tool had to compact is strictly worse than the same task started lean. Curating up front isn't just cheaper — it's more reliable than salvaging a giant session after the fact.

Chunk inputs that are too big to fit

Sometimes the thing you need the model to process is simply larger than its context window — an entire codebase, a giant log, a long document. You cannot pass it all at once; that's Limitation #1 in its most literal form. The answer is to split the input into pieces, process each piece with its own clean context, and then combine the results.

This is exactly the canonical example from the three limitations: to scan a codebase, you break the code into chunks because you can't feed it in whole. When you're building this into a product via the API, chunking becomes a concrete engineering pattern (how to split, how much to overlap, how to merge) — see Chunking & Retrieval for the dedicated treatment, and Patterns for LLM Apps for where it fits among the other building blocks. The same limitation drives both the interactive habit and the production pattern.

Keep separate concerns in separate threads

Don't run a frontend bug, a database migration, and a documentation rewrite in one conversation. Their contexts pollute each other — the model carries assumptions from one into the next, and the desk fills three times as fast.

Give each concern its own space:

  • Separate threads / sessions for unrelated tasks — the simplest version, available to everyone.
  • Forks / branches to explore a variation without contaminating the main line of work.
  • Git worktrees so multiple agents can work in genuinely isolated copies of the repo at once.
  • Subagents to run a noisy sub-task — a wide search, an exploratory read — in its own context and hand back only the conclusion, keeping the exploration debris off the main thread.

This both protects context quality and lets independent work happen in parallel. The mechanics — worktrees, parallel sessions, when to use which — are covered in Parallel Sessions and Worktrees.

Practical habits

  • Front-load the relevant, cut the rest. Give the files and facts the task needs; leave the rest in the cabinet.
  • Prefer pointing over scanning. Name the file or directory whenever you can.
  • Summarize before you switch tasks. A two-line "where we are" note lets you start clean without losing the thread.
  • Watch the length. If a session has been running a long time and quality is slipping, that's the symptom — refresh.
  • One concern per thread. Resist the urge to pile a second task onto a session that's already deep in something else.

Building this into a product? When the same context is reused across many calls — a long system prompt, a fixed document — prompt caching is the cost lever: the model reuses the already-processed prefix instead of paying for it every time. See Patterns for LLM Apps.

For non-developers: the same logic applies to non-code work. Researching a topic, then drafting a document from a clean summary of what you found, beats doing both in one ever-growing chat. Investigate, distill, then execute fresh.

Sources