Skip to content

AI limitations

AI is powerful, but it is not magic. Knowing where it breaks is the single thing that matters, what it does and doesn't do well.

Some techniques, tools, and workflows in this knowledge base may look complicated at first. Retrieval pipelines, project memory files, chunking strategies, etc. and can feel a bit confusing.

However, most exists to work around one of three fundamental limitations of today's LLM models. Once you can name which limitation a technique is fighting, you can reason about when to use it, how to adapt it, and when to ignore it, instead of memorizing recipes.

  1. Limited memory — the model can only hold so much in mind at once, and quality drops well before that limit.
  2. No real thinking, no compounding learning — the model is brilliant but amnesiac; it doesn't accumulate experience the way a person does.
  3. Outdated knowledge — the model's training stopped at a point in time, and it has never seen your private or just-changed code.

Limitation 1: Limited memory

A model's context window is everything it can "see" at one moment: your instructions, the conversation so far, the files or documents you've pasted in, and the response it's currently writing. Think of it as working memory, not long-term memory. It is finite — today's frontier models top out somewhere between roughly 200,000 and 1,000,000 tokens (a token is about three-quarters of a word). That sounds enormous, and it is large. But it is still a hard ceiling, and a large codebase, a year of chat history, or a pile of PDFs will blow past it.

The subtler and more important problem is what happens before you hit the ceiling. Quality does not stay flat right up to the limit and then fall off a cliff. It degrades gradually as the context fills up.

Two well-documented effects describe this:

  • "Lost in the middle." When the information a model needs is buried in the middle of a long input, the model is markedly worse at using it than when the same information sits near the beginning or the end. Performance traces out a U-shape: strong at the edges, weak in the middle. This held true even for models explicitly built for long contexts.
  • "Context rot." As the total input grows, models become less reliable even on simple tasks, and this happens well before the window is full. In controlled tests across many frontier models, a model with a 200,000-token window showed meaningful degradation at a fraction of that — the tokens are technically "in there," but they effectively get lost.

The takeaway: more context is not automatically better. Past a point, adding material actively makes the model dumber. Curating what goes into the window matters as much as how much fits.

The consequence for how you work: you must split work into pieces. The more focus, the better. You cannot hand a model an entire large codebase and ask it to "understand the repo." Instead you chunk it — break it into coherent units, index those units, and retrieve only the few that are relevant to the question at hand. The same instinct applies everywhere: summarize a long thread before continuing, give the model the three files that matter rather than all forty, and keep each task tightly scoped so the signal isn't drowning in noise.

Limitation 2: No real thinking, no compounding learning

This one is easy to misjudge because the model sounds like it's thinking. It is worth being precise about what it does and doesn't do.

The model does not truly brainstorm with you, weigh ideas against lived experience, or learn from its mistakes over time the way a colleague does. Each session starts from a blank slate. It will not remember the bug it caused you yesterday, the architectural decision you made last week, or the convention it got wrong an hour ago in a different conversation. It is, to put it bluntly, brilliant but amnesiac.

Contrast this with how humans get good at things. A person is slow per step — we read, forget, re-read, take a wrong turn, back up. But over months and years that experience compounds. We build intuition, taste, and a mental model of the system that makes the hundredth decision faster and better than the first. That compounding is one of the largest advantages humans still hold. The model has raw capability that often exceeds ours in the moment, but no compounding: its thousandth answer is no wiser than its first.

The consequence for how you work is a clean division of labor:

  • Humans supply what the model lacks: direction, judgment, taste, and accumulated context. You decide what is worth building and whether an answer is actually good.
  • Externalize the model's missing memory into files. Since the model won't remember, write things down where it can re-read them every session: a CLAUDE.md with project conventions, written plans before large changes, design notes capturing why a decision was made. These artifacts are the model's long-term memory, kept outside the model. (See Memory and project rules.)

This reframes a lot of "prompting" advice. You are not coaxing a mind that learns; you are re-briefing a sharp consultant who has never seen your project before, every single time.

Limitation 3: Outdated knowledge

A model is trained on a snapshot of data that ends at a particular date — its training cutoff. Anything that happened, changed, or was published after that date is simply not in the model. It also never saw anything private: your internal repositories, your unreleased product, the function you renamed five minutes ago.

Tools like web search and documentation lookups help — they let the model pull in fresh facts at the moment you ask. But it's important to see what they do and don't fix. They mitigate the consequence (the model can now answer about recent events) without touching the root cause (the model itself still doesn't know these things; it's reading them off a page you handed it). And no web search can reveal your private codebase or the change you made this morning that you haven't pushed.

The consequence for how we work: feed current facts in, and verify anything time-sensitive. If a library has likely changed since the cutoff, fetch its current docs and put them in context rather than trusting the model's recollection of an older API. In development work, tools like Context7 — an MCP server that pulls a library's up-to-date documentation straight into the model's context — automate exactly this. If a claim depends on "what's true right now," check it.

This limitation is also why the "best model" keeps changing. Every new release pushes the cutoff forward and shifts capabilities, so the model that was clearly best six months ago may no longer be. Don't anchor to a favorite; choose per task. (See Choosing models.)

The load-bearing principle

Here is the single idea to carry out of this page:

Any approach should be chosen and adapted based on how a given limitation will constrain the desired output.

Don't reach for a technique because it's popular. Ask which of the three limitations stands between you and the result you want, then pick the tool that addresses that limitation. The same problem under a different limitation calls for a different approach.

A few worked examples:

You want to… Limitation in the way Technique it points to
Have the model reason over a large codebase Limited memory — the whole repo can't fit, and stuffing it in causes context rot Chunk and index the code; retrieve only the relevant pieces per question
Keep a long, multi-week project coherent No compounding learning — the model forgets every session Externalize memory into files: a CLAUDE.md, written plans, design notes the model re-reads each time
Use a library that changed (or one the model may not know well) Outdated knowledge — the model's recollection predates the change Fetch the current documentation and put it in context; verify the API against today's docs

Notice that the technique falls out of the diagnosis. That's the whole method. (For the mechanics of keeping context clean, see Context management.)

Where to go next

Sources

  • Liu et al., Lost in the Middle: How Language Models Use Long Contexts, TACL 2024 — https://arxiv.org/abs/2307.03172
  • ACL Anthology entry for the same paper — https://aclanthology.org/2024.tacl-1.9/
  • Chroma Research, Context Rot: How Increasing Input Tokens Impacts LLM Performance — https://research.trychroma.com/context-rot
  • Anthropic, Context windows (Claude Platform Docs) — https://platform.claude.com/docs/en/build-with-claude/context-windows
  • Anthropic, Models overview (Claude Platform Docs) — https://platform.claude.com/docs/en/about-claude/models/overview