Skip to content

Verification and Review

An AI agent will tell you it's done. It will sound confident. That confidence is not evidence. The gap between "the agent says it works" and "it works" is where bugs ship, and closing that gap is a discipline you build into the loop — not a step you bolt on at the end.

Why this matters — Limitations #1 and #2. The model can't truly reason about whether its own output is correct (#2), and over a long task its attention drifts and it loses track of earlier constraints (#1). It is also agreeable by default — ask "did this work?" and it leans toward "yes." So verification can't be the model's self-assessment. It has to be external evidence: tests that run, checks that pass, and human judgment where it counts.

Make verification part of the loop, not an afterthought

Build verification into the work from the first step:

  • Ask for tests alongside the implementation. "Write the function and the tests that prove it works" — not "write the function" now and tests "later" (later rarely comes).
  • Have the agent actually run things. Tests, linters, type-checks, the build. Running them — and pasting the real output — turns a claim into evidence. "It should pass" is worthless; the green test run is the point.
  • Ask it to state what was and wasn't tested. "Which cases do these tests cover, and which did you not cover?" This surfaces the gaps the agent would otherwise paper over, and tells you exactly where your own attention is needed.

Tip: this is the best moment you'll ever get to finally adopt TDD (Test-Driven Development). Write the failing tests first, then tell the agent to make them pass. The tests become the spec the agent works against — it can't quietly drift, and "done" gets a concrete, external definition instead of the agent's say-so.

This pairs directly with Plan and Design First: small, independently testable chunks make verification easy because each step has a clear "does it work?" answer.

The key risk: agents weaken tests to make them pass

This deserves its own flashing sign.

WATCH OUT. When an agent is told "make the tests pass," a failing test is an obstacle — and one easy way to remove an obstacle is to weaken the test. Agents will delete assertions, loosen an exact match to "contains," add skip, widen an expected range until anything fits, or mock away the very thing under test. The suite goes green. Nothing actually got fixed. Always review test changes as carefully as production code — more carefully. A weakened test is worse than a failing one, because it lies to everyone afterward and hides the regression it was supposed to catch.

When you review a diff, look at the test changes first. If a test was modified, ask: did the behavior legitimately change, or did the agent just make the test stop complaining?

Layered review: agent → independent agent → human

No single reviewer catches everything. Stack three layers, each catching what the previous misses:

  1. The agent reviews its own diff. Cheap and worthwhile as a first pass — catches obvious leftovers, debug prints, missed cases. But it shares the blind spots of whatever wrote the code, so it's necessary, not sufficient.
  2. A second, independent agent reviews. A fresh agent — separate session, no memory of writing the code — reviews far more honestly than the author. It isn't invested in the approach and isn't carrying the author's assumptions. Running it in a separate thread or worktree keeps its judgment clean (see Context Management and Parallel Sessions and Worktrees). This is the strongest automated layer.
  3. A human reviews the final diff. The last line. The human owns the judgment calls no model should make alone (below).

What AI review is good at — and what humans must own

Use each reviewer for what it's actually good at.

AI review is reliable for… Humans must own…
Mechanical issues (style, dead code, leftover debug output) Architecture — is this the right structure?
Missing or thin test coverage Product fit — does this solve the real problem?
Obvious bugs and unhandled cases Security — trust boundaries, auth, secrets, attack surface (more)
Documentation and comment gaps Data correctness — is the data actually right?
Inconsistencies with stated conventions Performance at real scale and cost
Obvious redundancy and duplication Maintainability — will the team live well with this?
Should this exist at all?

The pattern: AI is excellent at "is this code mechanically sound?" and unreliable at "is this the right thing to build?" The further a question moves from the diff toward judgment, context, and consequences, the more it belongs to a human. Don't outsource the decisions you'll be accountable for.

A strong review prompt

When you ask any reviewer — the author agent, an independent agent, or yourself working with one — to review a diff, use a prompt that targets the things that actually go wrong. Keep this one handy:

Review this diff for redundancy, hidden regressions, architectural drift, missing tests, and places where the implementation is more complex than necessary. Give me a final checklist before merge.

It deliberately points at the failure modes agents are prone to — over-engineering, silent regressions, drift from the intended design, and thin tests — and ends by forcing a concrete, actionable checklist rather than a vague "looks good."

Automate the review where it fits

The independent-reviewer layer doesn't have to be manual. Two common ways to make it routine:

  • AI-assisted PR review. Wire an agent to comment on pull requests — it posts the mechanical findings (missing tests, redundancy, obvious bugs, doc gaps) inline so the human reviewer can spend their attention on the judgment calls. Claude Code ships a code-review capability for exactly this; see Skills and Plugins.
  • CI review agents. Run a review agent as a continuous-integration step that flags the PR (or fails the build) on regressions, weakened tests, or convention violations — the same checks above, enforced automatically on every change.
  • Tighten the CI. Make it a nightmare to get all green checks: linter & formatter throwing (no warnings), build should pass, static type checker, etc. The more, the better. Couple this with pre-commit hooks.

Automating the mechanical layer is high-leverage, but it doesn't replace the final human read of the diff. It clears the noise so human attention lands where it matters.

Quick checklist

  • Tests requested alongside the implementation, not deferred
  • Agent ran the tests / linters / type-checks / build and pasted real output
  • Agent stated explicitly what was and wasn't tested
  • Test changes reviewed first — confirmed they weren't weakened to pass
  • An independent agent reviewed the diff, not just the author
  • A human owned architecture, security, data, and "should this exist"

Sources