Skip to content

Two Ways To Use AI

Before reading anything else in this knowledge base, let's make one distinction clear. There are two fundamentally different ways to use AI and this is currently being mixed up with all the noise around it. Some confusions about "how should we do AI here" dissolve once you know which of the two you're talking about.

  1. AI inside a product — You call model APIs as a building block of software you ship to customers. You'll need robust prompts, supervised flows and orchestrations, likely things like structured outputs to ensure consistency, and some safety against prompt injection.
  2. AI as a development tool — You use AI assistants to build software faster, design faster, and overall get work done faster.

They share the same underlying technology, and both are shaped by the same limitations. But they involve different skills, different definitions of success, and different risks. The whole knowledge base is organized around this split, so it's worth understanding.

Way 1: AI inside your products (the API path)

Here, AI is a component of a product you build and ship. You write code that calls a model's API, and its output becomes part of how the application works.

Take AuditAgent, a smart-contract security tool: it sends carefully constructed prompts to a model, gives the model tools to call, manages what information goes into the limited context window, and turns the model's responses into results a user can act on. The AI is woven into the product's machinery.

The defining fact about this path: AI is a dependency of the product. That has consequences.

  • You design the AI usage in code. Prompts, the tools the model can call, how you store and recall "memory," how you chunk large inputs to fit the context window — all of it is engineered, version-controlled, and tested.
  • You pay per token. Every call costs money in proportion to how much text goes in and comes out. Cost is an ongoing operational concern, not a one-time license.
  • You own the hard non-functional stuff: reliability (it has to work when a customer hits it), latency (it has to be fast enough), evaluations (measure whether output quality is good and stays good as you change prompts or swap models), and safety (it must not do harmful or unintended things on real user data).

This is where the three limitations show up as engineering constraints you build around. Limited memory becomes a chunking-and-retrieval design problem. The lack of compounding learning becomes a "where do I store state between calls" design problem. Outdated knowledge becomes a "how do I feed in current and private data" design problem. The patterns for handling all of this live in Patterns for LLM apps.

In one line: building AI into a product is software engineering, where one of your components happens to be a probabilistic model you call over an API.

Way 2: AI as a development tool (assisted engineering)

Here, AI is a collaborator in your own workflow. You use tools like Claude Code or Codex to help you write, refactor, debug, and understand software faster. The AI never ships to the customer — it helps you produce the thing that ships.

The defining fact about this path: AI is a partner in how you work, not a part of what you deliver. The skill set is completely different from the API path. There's no token budget to engineer and no latency SLA to hit. Instead, the craft is:

  • Prompting — describing what you want clearly enough to get useful work back.
  • Context management — making sure the assistant is looking at the right files and information, and not drowning in irrelevant ones (this is the limited-memory limitation, felt directly in your day-to-day).
  • Planning — breaking work into pieces the assistant can do well, and deciding the order.
  • Review — reading what it produced with a skeptical eye and catching what's wrong. This is where your engineering judgment earns its keep.

If you're getting started on this path, see Getting started with Claude Code, and for the discipline of feeding it the right context, Context management.

In one line: using AI to build is a working relationship, where the craft is steering a capable but amnesiac collaborator.

Why conflating the two causes confusion

When people don't separate these two modes, they end up applying the instincts of one to the other — and it goes wrong in predictable ways:

  • Different skills. Being great at prompting Claude Code to refactor your service teaches you almost nothing about designing a reliable, evaluated prompt pipeline inside a shipped product — and vice versa. Someone strong in one can be a beginner in the other.
  • Different success metrics. For a product feature, success is measured in reliability, cost per request, latency, and evaluation scores on real traffic. For the development-tool path, success is simply: did it help you ship good code faster, and did you catch its mistakes in review?
  • Different risks. A bad output from your coding assistant gets caught in code review before it ever reaches anyone — the blast radius is small and contained. A bad output from the model inside your product can reach a customer directly, on their real data, at scale. The stakes and the safeguards are different.

A question like "how do we handle the model being wrong?" has two completely different right answers depending on which mode you mean. Name the mode first.

The two modes at a glance

AI inside your products (API path) AI as a development tool (assisted engineering)
What AI is A dependency of the product you ship A collaborator in your workflow
Who sees the output The customer, directly You — it never ships as-is
Example AuditAgent (smart-contract security tool) Claude Code / Codex helping you build
Core skills Prompt/tool/memory design in code, chunking, evaluations Prompting, context management, planning, review
Cost model Pay per token, ongoing operational cost Tooling/subscription cost; not per-feature
Success means Reliability, low cost, low latency, good eval scores, safety You shipped good code faster and caught the mistakes
Main risks Wrong output reaches customers at scale, on real data Wrong output slips past review into the codebase
The three limitations are… Engineering constraints you build around Daily friction you steer around

What transfers — and what doesn't

The two paths are distinct, but they aren't sealed off from each other. Some skills carry over; some don't. Knowing which is which tells you when experience in one mode is worth something in the other.

What transfers:

  • Both are bounded by the three limitations. Limited memory, no compounding learning, and outdated knowledge constrain a shipped product and a coding assistant alike. The instinct of asking "which limitation is in the way here?" works in both modes.
  • Prompting. Describing a task clearly, giving the model the right framing and examples, and getting structured output back is a skill that pays off whether you're prompting Claude Code or writing the prompt your product sends at runtime.
  • Model selection. Judging which model fits a task — strong reasoning vs. fast-and-cheap — matters when you pick the model behind a product feature and when you pick the model driving your editor. (See Choosing models.)

What doesn't transfer:

  • Cost and latency engineering is specific to the product path. Token budgets, caching, batching, latency SLAs, and evaluation pipelines on live traffic have no equivalent when AI is just helping you write code.
  • Personal workflow discipline is specific to the development-tool path. Context hygiene, plan-first habits, and review discipline are about how you work — they don't ship inside a product and don't substitute for the engineering rigor the API path demands.

In short: the understanding of the model carries across; the engineering and habits built around it are mode-specific. For the development-tool craft, the Foundations and Workflows sections go deep; for the product craft, see Integrating AI via API.

Where to go next