Most AI systems are built from a small set of recurring shapes. Knowing them means you reach for the simplest one that solves the problem, instead of defaulting to a fully autonomous agent for a job a straight pipeline could do more cheaply and reliably.
A model call is probabilistic, not deterministic — the same input can produce a slightly different output twice in a row. It's also slow and costly relative to normal code, and it can fail in ways a function call can't: a confidently wrong answer, a malformed tool call, a hallucinated fact.
Architecture is how you contain that. The right pattern breaks a big, fuzzy task into smaller pieces the model is reliably good at, adds checks where mistakes are expensive, and keeps the system's cost and latency proportional to how hard the task actually is.
These range from simplest to most autonomous. Start from the top and only move down the list when a simpler pattern genuinely can't do the job.
Break one big task into an ordered sequence of smaller model calls, each working from the previous one's output. Best for a task that decomposes cleanly into fixed stages — draft, then check facts, then adjust tone.
Classify the input first, then send it down a specialized path built for that category. Best when requests fall into distinct types that each deserve a different prompt, model, or tool set.
Run several model calls at once on the same input — either splitting the work into sections, or running the same prompt multiple times and voting on the results. Best for cutting latency or improving reliability through consensus.
A central model breaks an open-ended task into sub-tasks it can't fully predict in advance, and dispatches each to a worker. Best when the number and shape of sub-tasks genuinely depends on the specific input.
One model generates a response; a second call — or the same model in a different role — critiques it against explicit criteria and sends it back for revision. Best when quality is easy to judge but hard to get right on the first attempt.
Look up relevant information from an external source before generating an answer, instead of relying only on what the model already knows. Best when answers must be grounded in specific, current, or private data.
If you can describe it as "always do A, then B, then C," you likely need nothing more than prompt chaining — don't reach for autonomy you don't need.
If requests cleanly split into a few types that need different handling, add a routing step before you build separate pipelines for each.
If sub-tasks don't depend on each other's output, parallelize them instead of running them one after another.
If you can't know ahead of time how many pieces a task breaks into, that's the signal for an orchestrator-workers setup rather than fixed routing.
Tasks like precise translation or code that must pass tests benefit from an evaluator-optimizer loop layered on top of whichever pattern generates the draft.
Private documents, current data, or anything post-dating training data calls for retrieval — add RAG regardless of which control-flow pattern you land on.
Reach for an open-ended tool-calling loop when the path truly can't be predetermined by any of the above — it's the most flexible pattern and also the hardest to test and the most expensive to run.
Capture intermediate prompts, tool calls, and results. When something goes wrong, you need to see which stage produced the bad output, not just that the final answer was wrong.
A small, representative set of test cases with expected outcomes lets you tell whether a prompt or architecture change actually helped, instead of guessing from a handful of manual tries.
Anything that spends money, deletes data, or goes out to a real person should have an approval gate or a strict validation check, regardless of how reliable the model has seemed so far.
Cap how many tool calls or retries a single task can make. Without a ceiling, a stuck loop or a runaway agent can burn far more time and money than the task warrants.
Treat prompt and system-message changes as reviewable, revertible changes — track what changed and be able to roll back a regression as easily as you'd revert a bad commit.
Structured, clearly labeled tool results are easier for a model to reason over correctly than a dense human-oriented report — the model is a consumer of that data too.
Decide up front what happens on a malformed tool call, a timeout, or a low-confidence answer — a fallback path or a graceful failure message, not an unhandled exception.
A fixed, five-step task doesn't need autonomy — it needs prompt chaining. Autonomy adds cost, latency, and unpredictability you don't get anything back for.
Cramming instructions, formatting rules, and edge cases into a single mega-prompt makes failures hard to diagnose. Smaller, single-purpose steps are easier to test and fix independently.
A RAG system that never re-indexes will confidently answer from stale documents — treat the index's freshness as a monitored property of the system, not a one-time setup step.
If one model critiques its own output using the same assumptions that produced it, it will miss the same mistakes. Vary the model, the prompt, or the criteria used for evaluation.