Skip to content

Tool Loops & Agents

A single LLM call takes a fixed input and returns a fixed output. That's enough for "summarize this" but not for "audit this codebase," where the model needs to look things up as it goes — read a file, run an analyzer, follow a lead, decide what to do next. The pattern that enables this is the tool-use loop, and an "agent" is essentially a tool loop with enough autonomy to choose its own path.

For non-engineers: a tool loop turns the model from someone answering a single question into someone who can say "let me check something" and come back with a more informed answer. You give it a set of actions it's allowed to take (search files, read a function, run a test), and it decides which to use and when to stop.

Server tools vs client tools

Tools come in two flavors, and the distinction matters:

  • Server tools run inside the model provider. You declare them, the provider executes them, and the result is folded into the response transparently. Anthropic's built-in web search is a server tool — your application never sees the call.
  • Client tools run in your own code. The model emits a request ("call search_files with query X"), your runtime invokes the matching function, and you send the result back as part of the next turn. You own the function and its lifecycle — this is where most real product logic lives.

The loop

The mechanics are simple and always the same shape:

1. Call the model with the conversation + the list of available tools.
2. Look at the response:
     - If it contains tool-use requests → run the matching handlers,
       append the results to the conversation, go back to step 1.
     - If it contains a final answer → return it.
3. Guard: if a turn budget is exhausted → make one final call with NO tools
   to force a plain answer, then return.

AuditAgent implements exactly this in its complete_structured loop. A few details that make it production-grade:

  • The loop ends on a typed answer or a budget cap. When the model returns a parsed, schema-valid response (see Structured output & verification), you're done. If it keeps calling tools past a max_turns limit, one final tool-free call coerces an answer rather than looping forever.
  • Parallel tool calls. If the model requests several independent tools in one turn, run them concurrently.
  • Config is preserved across iterations — reasoning effort, timeouts, retry behavior stay consistent turn to turn.

You often don't have to hand-roll this loop. The official SDKs ship a tool runner (currently in beta) that drives the whole cycle for you: it calls the model, runs your tool functions, feeds the results back, and stops when the model is done. Reach for it when you want the default behavior; write the loop by hand (as above) when you need fine-grained control — approval gates, custom logging, conditional execution. See Anthropic's tool use docs.

Writing a good tool definition

A client tool has three pieces:

  1. A definition — name, description, and an input schema (JSON Schema) describing each argument.
  2. A handler — the async function that runs when the model calls it, returning a string the model will read.
  3. A binding that pairs them, usually with a per-call timeout.

The single highest-leverage thing here: the description is the only thing the model sees. Call quality is directly determined by description quality. A good one is 3–5 sentences covering what the tool does, when to use it, what each argument means, and what the output looks like. AuditAgent's search_files tool, for example, doesn't just say "searches files" — it says it's for "verifying a finding requires locating call sites or specific code constructs that are not already in the conversation," and documents that it returns a JSON array of {file, line} matches. That precision is what makes the model call it correctly.

# Illustrative — a client tool the model can call to look up code on demand.
SEARCH_FILES_TOOL = Tool(
    name="search_files",
    description=(
        "Search the repository under review for files containing a query. "
        "Useful when verifying a finding requires locating call sites or "
        "specific code constructs not already in the conversation. "
        "Optionally filter by file extension. Returns a JSON array of "
        "{file, line} matches, sorted by file path."
    ),
    input_schema={
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "Substring or pattern to search for."},
            "extension": {"type": "string", "description": "Optional extension filter, e.g. 'sol'."},
        },
        "required": ["query"],
    },
)

async def search_files(*, query: str, extension: str | None = None) -> str:
    matches = await search_repo(query, extension=extension)
    return json.dumps([{"file": m.path, "line": m.line} for m in matches])

Agentic exploration: letting the model drive

A tool loop becomes an agent when you stop pre-selecting what it looks at and instead hand it a toolbelt to navigate on its own. AuditAgent's agentic strategy does this: instead of structurally choosing which code to analyze, it gives the model layered navigation tools and lets it explore like a human auditor would:

  • list_modules, get_module_skeletons, get_skeleton(file) — zoom from map → outline → detail
  • read_file, read_method, investigate_method (reads a method + its callers + callees in one round-trip)
  • get_callers, get_callees, search_code (ripgrep-backed)
  • write_notes / read_notes — a scratchpad the agent uses to track hypotheses
  • report_finding — the only way a result escapes the loop

This is powerful but has to be bounded, or it will explore (and bill) forever.

Guarding against runaway loops and cost

Autonomy without limits is a cost and reliability hazard. The guards AuditAgent uses are worth copying:

  • Turn and tool-call budgets. The agentic loop caps max_turns (e.g. 80) and max_tool_calls (e.g. 100–150 depending on tier). When the budget runs out, the loop ends.
  • A single escape hatch for output. Findings only count if emitted via report_finding. The model can't accidentally "return" half-formed work.
  • Tiered budgets. Cheaper product tiers get smaller budgets; the expensive, unbounded exploration is reserved for the highest tier. Clients pick a tier; the budget is derived server-side so they can't crank it themselves.
  • Per-tool timeouts, so one slow tool can't hang the whole loop.
  • Isolated "lanes." Independent investigations get their own focus and scratchpad, so they don't pollute each other's context.

Cost guard rule of thumb: every agentic feature needs an answer to "what's the worst case?" before it ships. Multiply max turns × tool calls × token cost and make sure the ceiling is acceptable. An agent with no budget cap is a budget incident waiting to happen.

Tool safety & permissioning

The model is choosing which of your functions to run. Treat tool exposure like an API surface:

  • Least privilege. Only expose tools the task needs. AuditAgent's content-merge agent, for instance, runs under a hard permission cap that denies any write outside a single allowed file — even though the agent could in principle request others.
  • Validate arguments in the handler; never trust the model to stay in bounds.
  • Run in isolation when tools touch the filesystem or run commands (AuditAgent runs these in isolated scan pods).
  • Read vs. write asymmetry. Read tools are low-risk; write/execute tools need real guardrails.

Takeaways

  • Tool loops turn one-shot calls into iterative, look-it-up-as-you-go problem solving.
  • The tool description is the interface the model programs against — invest in it.
  • "Agentic" = the model picks its own path. Powerful, but always bound it with turn/tool budgets and a single output path.
  • Treat exposed tools as a security surface: least privilege, validated args, isolation for anything that writes or executes.
  • Know your worst-case cost before shipping any agentic feature.

Sources