Agent Architectures

An "agent" is an LLM running in a loop, choosing tools and acting on what comes back, until it decides it's done or a limit stops it. Almost every AI engineering interview now has an agent question, either conceptual ("ReAct vs plan-and-execute?"), design-shaped ("design a research agent / support agent / coding agent"), or skeptical ("why not just a workflow?"). This page covers the whole design space: the workflow patterns that should come first, the single-agent loop and its reasoning variants, planning, the main multi-agent topologies and the 2025–2026 argument over when they pay off, plus state, durability, protocols, frameworks and failure modes. The goal is that you can pick an architecture for a given problem and defend it with mechanisms and numbers.

1. What an agent is

Here's the working definition most practitioners use: an agent is a system where an LLM dynamically directs its own process and tool usage, keeping control over how it accomplishes the task. A workflow is one where LLMs and tools are orchestrated through predefined code paths. Anthropic 2024 Both are "agentic systems". What separates them is who decides the control flow: your code or the model.

Mechanically, an agent has four parts:

The basic building block is the augmented LLM: one model call that can use retrieval, tools and memory. Anthropic 2024 Every pattern below is a composition of augmented-LLM calls with different control flow around them.

The autonomy spectrum

"Agent or not" is the wrong binary. Think of a dial for how much control flow the model owns:

2. Workflows vs agents: the five workflow patterns

Anthropic's "Building effective agents" post (Dec 2024) became the shared vocabulary for this. It lists five workflow patterns and then the autonomous agent as the sixth, most flexible option. Anthropic 2024 Interviewers expect you to know the names and when each one fits.

The patterns, mechanically

Prompt chaining. Break the task into fixed sequential steps; each LLM call consumes the previous output. Add programmatic gates between steps (schema validation, length check, a classifier) so a bad intermediate result fails fast instead of contaminating later steps. You spend latency to get accuracy: each call does a simpler job. Use it when the decomposition is known and stable (generate marketing copy → translate it; write an outline → check it against criteria → write the doc).

Routing. Classify the input, then dispatch to a specialized prompt, toolset or model. This gives separation of concerns: optimizing the refund prompt can't degrade the tech-support prompt. It also handles cost routing (easy queries to a small model, hard ones to a frontier model). The router can be an LLM, a fine-tuned classifier or an embedding nearest-neighbor. Routing errors are silent, so log the route and evaluate the router on its own.

Parallelization. There are two flavors. Sectioning splits a task into independent subtasks that run concurrently (one call answers the user while another screens for policy violations; one call per document section). Voting runs the same task several times and aggregates. That's self-consistency applied at the system level Wang+ 2022, useful for reviews ("flag if any of 3 reviewers flags") or high-stakes classification. With sectioning, latency is the slowest branch instead of the sum. With voting, cost scales linearly with the number of votes.

Orchestrator–workers. A central LLM decomposes the task at runtime, delegates subtasks to workers, and synthesizes the results. The difference from parallelization is that the subtasks aren't predefined; they depend on the input (which files need changing, which sub-questions need researching). This is already a small multi-agent system, and §6 develops it further.

Evaluator–optimizer. One call generates, another evaluates against explicit criteria and returns feedback, and the loop repeats until it passes or hits a max-iteration cap. It works when (a) clear evaluation criteria exist and (b) feedback measurably improves the output, the way a human editor's would. Literary translation, code that must pass tests, and search tasks that need several rounds are good fits. It's the workflow version of Self-Refine Madaan+ 2023 and Reflexion (§4).

3. Anatomy of the agent loop

Every agent framework, however much it abstracts, implements something like the loop below. You should be able to write it on a whiteboard and point out where each production concern lives.

def run_agent(task, tools, llm, limits):
    messages = [system_prompt(tools), user(task)]
    state = State(steps=0, tokens=0, cost=0.0, started=now(), seen_actions=Counter())

    while True:
        # ---- budget / stop checks (harness-enforced, NOT model-enforced) ----
        if state.steps >= limits.max_steps:         return finalize(messages, reason="max_steps")
        if state.cost  >= limits.max_cost_usd:      return finalize(messages, reason="budget")
        if now() - state.started > limits.timeout:  return finalize(messages, reason="timeout")
        if context_tokens(messages) > limits.ctx_soft_cap:
            messages = compact(messages)            # summarize / drop old tool outputs

        # ---- THINK: model proposes next action(s) ----
        resp = llm(messages, tools=tools)           # may contain text + 0..n tool calls
        state.update_usage(resp.usage)
        messages.append(assistant(resp))

        if not resp.tool_calls:                     # model chose to answer
            return resp.text                        # (or require an explicit finish() tool)

        # ---- ACT: execute tool calls (parallel if independent) ----
        results = []
        for call in resp.tool_calls:                # or asyncio.gather for parallel calls
            if not policy.allows(call):             # permissions / HITL approval
                results.append(tool_error(call, "denied by policy")); continue
            state.seen_actions[hash(call)] += 1
            if state.seen_actions[hash(call)] > 2:  # loop detection
                results.append(tool_error(call, "repeated identical call; try something else")); continue
            try:
                out = tools[call.name].run(validate(call.args), timeout=limits.tool_timeout)
                results.append(tool_result(call, truncate(out, limits.max_obs_tokens)))
            except Exception as e:                  # errors go BACK to the model as observations
                results.append(tool_error(call, str(e)))

        # ---- OBSERVE: feed results back; loop ----
        messages.extend(results)
        state.steps += 1
        checkpoint(state, messages)                 # durability / resumability

Stop conditions and budgets

An agent ends for one of two kinds of reasons: it says it's done (natural), or the harness stops it (forced). You need both.

The quadratic cost of loops

Each step resends the whole history. If every step adds about \(\Delta\) tokens (thought + tool call + observation) on top of a base prompt \(P\), then step \(t\) has input \(P + t\Delta\), and total input tokens over \(n\) steps are

Back-of-envelope: \(P = 5\text{k}\), \(\Delta = 2\text{k}\), \(n = 30\) gives \(150\text{k} + 870\text{k} \approx 1\text{M}\) input tokens for one task. That's why (a) prompt caching matters so much for agents, since the stable prefix is re-read at a large discount; (b) truncating or summarizing tool observations pays off (\(\Delta\) is usually dominated by tool output); and (c) subagents can be cheaper per useful token than one long thread, because their large intermediate observations never enter the parent's context.

4. Single-agent reasoning patterns

These patterns differ in when the model reasons relative to acting, how much it plans ahead, and whether it explores alternatives. Most came from 2022–2024 papers. Today's models have absorbed many of them natively (reasoning models "think" between tool calls; frontier models emit parallel tool calls), so treat them as design vocabulary rather than recipes you have to implement by hand.

ReAct: interleaved reasoning and acting

ReAct prompts the model to alternate Thought → Action → Observation steps, so reasoning traces guide which tool to call next and observations ground the following reasoning. Yao+ 2022 The paper showed that combining the two beat reasoning-only chain-of-thought (which hallucinates facts) and acting-only (which can't plan or recover) on QA/fact-checking and interactive tasks. ReAct is the default shape of essentially every tool-calling loop today. Native function calling replaced the text "Action:" parsing, and reasoning models put the "Thought" in hidden thinking tokens.

• Strengths: adaptive, since each step uses the latest observation; robust to surprises; simple.
• Weaknesses: one LLM call per action (latency, cost); greedy, with no global plan, so it can wander or loop; history grows quadratically in cost.

Plan-and-execute / Plan-and-Solve

First generate a full plan (a list of steps), then execute the steps, often with a cheaper executor model or plain code, and optionally replan after each step or on failure. Plan-and-Solve prompting showed that asking the model to "first devise a plan, then carry it out" reduces missing-step errors compared with zero-shot CoT. Wang+ 2023 In agent systems the pattern is: a planner (strong model) emits steps; an executor (ReAct sub-loop, or a smaller model) runs each one; a replanner looks at results and revises the remaining steps.

• Strengths: global view of the task; a big model is called rarely; the plan is inspectable (and human-approvable) before execution.
• Weaknesses: plans go stale when early observations contradict assumptions, so you need a replanning policy; over-planning trivial tasks wastes tokens.

ReWOO: reasoning without observation

ReWOO goes further. The planner writes the whole plan with variable placeholders for tool outputs (e.g. #E1 = Search[x]; #E2 = LLM[summarize #E1]), workers fill in the evidence, and a solver composes the answer. Because the planner never re-reads intermediate observations, it avoids resending the growing history. The paper reports about 5× token efficiency and a 4% accuracy gain on HotpotQA. Xu+ 2023 The cost is zero adaptivity: if #E1 comes back empty, the plan doesn't react.

LLMCompiler: plans as parallel DAGs

LLMCompiler borrows from classical compilers. A Function Calling Planner emits a DAG of tool calls with dependencies, a Task Fetching Unit dispatches calls as soon as their inputs are ready, and an Executor runs independent calls in parallel. Against ReAct it reports up to 3.7× latency speedup, 6.7× cost savings and ~9% accuracy improvement. Kim+ 2023 It also supports replanning when results require it. Modern APIs' parallel tool calls (several tool calls in one model turn) are a lightweight, model-native version of the same idea, limited to one "layer" of the DAG per turn.

Reflexion and self-critique

Reflexion lets an agent learn across trials without weight updates. After a failed attempt, it writes a verbal reflection ("I failed because I didn't check edge case X"), stores it in an episodic memory buffer, and conditions the next attempt on it. It reported 91% pass@1 on HumanEval against 80% for the GPT-4 baseline at the time. Shinn+ 2023 Self-Refine is the single-trial version: generate → self-feedback → refine. Madaan+ 2023

Tree search: Tree of Thoughts and LATS

Tree of Thoughts (ToT) generalizes CoT into search over "thoughts" (coherent intermediate steps). The model proposes several candidates per step, evaluates them (value prompt or vote), and runs BFS/DFS with backtracking. On Game of 24, GPT-4 with CoT solved 4% of tasks versus 74% with ToT. Yao+ 2023 LATS (Language Agent Tree Search) brings Monte Carlo Tree Search to acting agents: LM-generated value estimates, self-reflections, and real environment feedback guide expansion. It reported 92.7% pass@1 on HumanEval with GPT-4. Zhou+ 2023

The catch is cost: search multiplies LLM calls by the branching factor × depth, often 10–100× a single ReAct run. It also needs either reversible environments (you can't "backtrack" a sent email) or simulation. In 2026 practice, explicit tree search is rare in products. Its role has largely moved into reasoning models' internal thinking and into best-of-n with a verifier (e.g. run n coding attempts in parallel sandboxes, pick the one that passes tests).

CodeAct: code as the action space

Instead of one JSON tool call per turn, the agent writes executable Python that can call multiple tools, loop, branch, and store intermediate values in variables. The interpreter's output (including tracebacks) is the observation. Across 17 LLMs, CodeAct outperformed JSON/text action formats with up to 20% higher success rate. Wang+ 2024 Hugging Face's smolagents makes CodeAgent its first-class agent type, with sandboxed execution via E2B, Modal, Docker and others. HF smolagents docs

# One CodeAct step replaces ~N JSON tool-call turns:
results = [search(f"{city} population 2025") for city in ["Tokyo", "Delhi", "Shanghai"]]
pops = {c: parse_number(r) for c, r in zip(["Tokyo", "Delhi", "Shanghai"], results)}
print(max(pops, key=pops.get), pops)      # only this printed summary enters context

Why it works: code is composable (loops and function nesting), models have seen huge amounts of it in pretraining, and intermediate data stays in interpreter variables instead of flowing through the context window. Costs: you need a real sandbox (arbitrary code execution is a security boundary), and failures are harder to attribute than with typed tool calls. The same idea appears as "code execution with MCP" or programmatic tool calling in vendor APIs, where the model writes code that calls tools so large intermediate results never enter context.

5. Planning in depth

"Planning" in agents covers three separate decisions: when to plan (upfront vs interleaved), how to represent the plan, and how deep to decompose (flat vs hierarchical).

Upfront vs interleaved replanning

Plan representations

Anthropic's long-running-agent harness (Nov 2025) is a good concrete example of "plan as files". An initializer agent runs once to set up the environment: an init.sh, a claude-progress.txt log, an initial git commit, and a JSON feature list where every feature starts with a "passes" status of false. Each later coding agent session reads the progress file and git log, picks one feature, implements it, verifies it end to end (browser automation), commits, and updates progress. Anthropic 2025 The plan lives outside any single context window, so the work survives context resets.

Hierarchical decomposition

For big tasks, decompose recursively until the leaves are "one agent, one context window, verifiable". Rules of thumb:

• Decompose by independence, not by noun. Good subtasks have minimal shared state and a clear output contract.
• Each leaf needs a done-criterion the executor can check (test passes, N sources found, schema-valid output).
• Depth costs context: each level of delegation is a lossy compression of intent. Two levels is common; more than three rarely pays off.

When does planning help?

• Helps: long-horizon tasks (more than about 10 steps), decomposable tasks with parallelizable parts, tasks where a human should approve the approach, and tasks where the agent must manage progress across context windows.
• Hurts or wastes effort: short tasks (planning overhead is larger than the task), highly reactive environments (plans go stale immediately), and tasks where the model's first plan is confidently wrong and nothing triggers a replan.
• 2026 nuance: reasoning models do a lot of implicit planning in their thinking tokens, so explicit planning scaffolds give smaller gains on short tasks than they did in 2023. The persistent, externalized plan (todo/progress file) is still clearly valuable for long tasks, mainly as a memory and anti-drift device.

6. Multi-agent architectures

A "multi-agent system" is any system where more than one LLM loop, each with its own prompt, tools and (usually) its own context window, cooperates on a task. The reasons to split are always some mix of: context isolation (each agent sees only what it needs), parallelism (wall-clock speed, more total tokens spent on the problem), specialization (different prompts/tools/models per role), and organizational modularity (teams own different agents). The costs are always: lost context at boundaries, coordination overhead, more tokens, and harder debugging.

(a) Orchestrator–worker / lead agent with subagents

A lead agent plans, spawns subagents with specific subtasks (often in parallel), receives their condensed results, and synthesizes. Subagents usually can't talk to each other. Everything goes through the lead.

The reference implementation is Anthropic's Research feature (June 2025). A lead agent (Claude Opus 4) plans, saves the plan to memory (in case the context gets truncated), spawns subagents (Claude Sonnet 4) that search in parallel, then synthesizes, with a separate CitationAgent pass to attribute sources. Anthropic 2025 The reported numbers, worth memorizing with their caveats:

• The multi-agent system outperformed single-agent Opus 4 by 90.2% on Anthropic's internal research eval. This is an internal benchmark of breadth-first queries, not a general result.
• Token usage alone explained ~80% of performance variance on BrowseComp, with tool-call count and model choice making up the rest of the roughly 95% explained. This is the key mechanistic insight: much of multi-agent's advantage is that it spends more tokens in parallel, across more context windows than one agent could hold.
• Agents used about 4× the tokens of chat interactions, and multi-agent systems about 15×.
• The lead typically spawned 3–5 subagents in parallel, and subagents used 3+ tools in parallel. Together these cut research time by up to 90% for complex queries.

Engineering lessons from that post that come up in interviews: teach the orchestrator to scale effort to query complexity (simple fact-finding gets 1 agent with a few tool calls; complex research gets many subagents); give subagents detailed task descriptions (objective, output format, tool guidance, boundaries), because vague delegation caused duplicated work and gaps; use "start wide, then narrow" search strategies; and have subagents write outputs to a filesystem/artifact store and pass back lightweight references, so large content doesn't get squeezed through the lead's context. Anthropic 2025

(b) Supervisor / router

A supervisor agent decides, at each turn, which specialist agent acts next (or routes the whole request once). Specialists return control to the supervisor after they act. It differs from orchestrator–worker mostly in emphasis: the supervisor is a dispatcher over a fixed roster of specialists (often sequential), whereas the orchestrator creates subtasks dynamically (often parallel). In OpenAI's Agents SDK terms this is the "agents as tools" / manager pattern: the manager calls specialists as tools and keeps ownership of the final answer. OpenAI docs

• Pros: central control and a single place for policy; easy to add specialists; one coherent voice to the user.
• Cons: an extra LLM hop per turn (latency, cost); the supervisor's routing quality caps the system; if specialists return verbose outputs, the supervisor's context bloats.

(c) Hierarchical teams

Supervisors of supervisors: a top-level coordinator delegates to team leads (e.g. "research team", "writing team"), each managing their own workers. It mirrors org charts and is how MetaGPT-style "software company" simulations are structured, with roles like product manager, architect and engineer following SOPs. Hong+ 2023

• Pros: scales to large task trees; each supervisor's span of control stays small; teams can be developed and evaluated separately.
• Cons: every level is a lossy re-statement of intent (the telephone game); latency multiplies with depth; hard to trace failures to a level. Rarely justified beyond two levels.

(d) Handoffs / swarm

Decentralized transfer of control: the active agent decides to hand the conversation to another agent, which then becomes the active agent and talks to the user directly. There's no central supervisor in the loop. OpenAI's experimental Swarm library introduced this pattern, and its README now says it's replaced by the OpenAI Agents SDK, "a production-ready evolution of Swarm". OpenAI Swarm In the Agents SDK, handoffs are exposed to the model as tools: a handoff to an agent named "Refund Agent" appears as a tool named transfer_to_refund_agent. By default the receiving agent sees the full conversation history, and an input_filter can trim what it sees. OpenAI Agents SDK docs

• Pros: no extra supervisor hop (lower latency); each specialist has narrow instructions and tools; natural for customer-service triage (triage → billing → refunds).
• Cons: no global view, so agents can ping-pong; harder to enforce cross-cutting policy; the full history passed along can confuse the receiver or leak irrelevant context; poor at multi-domain requests that need synthesis. LangChain's comparison notes handoffs are efficient for single-domain and repeat requests but weak on multi-domain queries. LangChain docs

(e) Sequential pipelines / assembly line

Fixed stages, each an agent (or LLM call) with its own role: researcher → writer → editor → fact-checker, or localize → repair → validate for code. This is really prompt chaining where the stages happen to be agentic. Agentless is a well-known example: a fixed three-phase localization → repair → patch-validation process with no LLM-chosen control flow. It scored 32% on SWE-bench Lite at about $0.70 per issue, beating the open-source agents of the time. Xia+ 2024

• Pros: predictable, testable per stage, easy to observe and cost-bound; each stage can use a different model.
• Cons: no backtracking unless you add explicit feedback edges; errors in early stages propagate; rigid for inputs that don't fit the pipeline.

(f) Debate / multi-agent critique

Several agents (same or different models, possibly with assigned stances) propose answers, read each other's reasoning, and revise over rounds until they converge or a judge decides. Du et al. showed that multi-agent debate improved math and strategic reasoning and reduced factual errors. Du+ 2023 The generator–verifier loop is the practical cousin: a reviewer agent with a clean context checks the producer's output.

• Pros: catches errors a single pass misses, especially when the critic has fresh context or is a different model family (decorrelated errors).
• Cons: costs rounds × agents × tokens; agents tend to converge on a confident wrong consensus (conformity), so the gain over plain self-consistency voting is contested; needs a stopping rule.

(g) Blackboard / shared state

A classic AI architecture: agents don't message each other. They read from and write to a shared workspace (the "blackboard"), and a controller (or the agents themselves) decides who acts next based on the blackboard's state. In LLM systems the blackboard is a shared document, a structured state object (LangGraph's graph state is effectively this), a database, or a repo.

• Pros: loose coupling (agents only know the shared schema); everything is inspectable; agents can be added without rewiring messages; natural for incremental, opportunistic problem solving.
• Cons: concurrent writes need conflict resolution (locks, CRDTs, versioned updates, reducers); the blackboard grows and needs curation; agents can overwrite each other's work, which is exactly Cognition's "conflicting implicit decisions" problem.

(h) Market / auction, peer-to-peer and network topologies

Market/auction (contract-net): a manager announces a task, agents bid (with estimated cost, confidence or capability), and the best bid wins. It's useful when agents are heterogeneous (different costs/tools) and the roster is dynamic, e.g. cross-organization agents discovered via A2A agent cards. Peer-to-peer / network: any agent can message any other (AutoGen-style group chats, CAMEL role-playing Li+ 2023). It's flexible, but communication grows as \(O(n^2)\), it's hard to reason about termination, and it's mostly a research or simulation setup. Generative Agents (the "Smallville" simulation) is the famous example of agent societies used for simulation rather than task completion. Park+ 2023

Summary table

7. Agent roles and how they differ

Role names get used loosely in interviews and papers. Here's a precise vocabulary. One agent can play several roles, and a role can be plain code instead of an LLM.

8. The case against vs the case for multi-agent

This was the defining architecture debate of mid-2025, and interviewers love it because it tests judgment rather than recall. The two key posts came out one day apart in June 2025.

The synthesis

Read the two posts together and they mostly agree. LangChain's commentary at the time framed it as reading vs writing: multiple agents gathering information (reading) parallelize well, while multiple agents producing parts of one artifact (writing) create conflicting implicit decisions that are hard to merge. Chase 2025 Cognition's own April 2026 follow-up, "Multi-Agents: What's Actually Working", lands on the same principle: writes stay single-threaded, and additional agents contribute intelligence rather than actions. The patterns it reports working are a clean-context code-review agent (generator–verifier), a "smart friend" frontier model consulted by the primary agent, and manager agents that coordinate scoped child agents. Parallel-writer swarms still don't work well. Cognition 2026

Controlled research points the same way. A Dec 2025 study by Google researchers and collaborators evaluated 260 configurations across six agentic benchmarks and five architectures (single-agent; independent, centralized, decentralized and hybrid multi-agent). Multi-agent performance relative to single-agent ranged from +80.8% on decomposable financial reasoning to −70.0% on sequential planning. Coordination showed diminishing returns once the single-agent baseline was already strong, tool-heavy tasks paid a multi-agent overhead, and architectures without centralized verification propagated errors more. Kim+ 2025 The MAST failure taxonomy (1,600+ annotated traces across 7 multi-agent frameworks) groups failures into system design issues, inter-agent misalignment, and task verification, and notes that multi-agent gains on popular benchmarks are often minimal. Cemri+ 2025

9. Inter-agent communication and handoff context

Agents communicate in three basic ways, and the choice decides what context survives a boundary.

What to pass in a handoff

Most multi-agent failures are really specification failures at the boundary. Every subagent starts with zero knowledge except its prompt. A good structured task spec looks like this:

{
  "objective": "Find 2024-2026 benchmark results for open-weight models on SWE-bench Verified",
  "why": "Parent is writing a comparison table for a CTO memo; needs numbers + sources",
  "scope": {"include": ["official leaderboards", "model cards"], "exclude": ["blog speculation"]},
  "known_context": ["Parent already has results for closed models; don't repeat them"],
  "constraints": {"max_tool_calls": 15, "max_tokens_out": 1200, "deadline_s": 120},
  "output_format": {"type": "json", "schema": "[{model, score, date, source_url}]"},
  "done_when": ">= 5 models with primary-source URLs, or budget exhausted (report gaps)",
  "decisions_already_made": ["Use 'Verified' split only", "Report pass@1"]
}

Notice why and decisions_already_made. They're the direct mitigation for Cognition's "implicit decisions" problem: you write down the decisions the subagent would otherwise make differently. Also ask for condensed outputs with references (findings + source pointers, or a file path) rather than raw transcripts, so the parent's context stays clean.

Returning results

• Summaries, not transcripts. A subagent might burn 50k tokens exploring but should return 1–2k tokens of distilled findings. That compression is the context-isolation benefit.
• Include uncertainty and gaps: "couldn't find X", "sources disagree on Y". Otherwise the parent assumes completeness.
• Structured outputs (JSON schema) make synthesis and validation mechanical.
• Large artifacts by reference: write to a file or object store and return the path plus a short summary.

10. State management and control flow

Once an agent does more than one tool loop, you need to answer: where does state live, who decides the next step, and what happens if the process dies or a human must approve something? The dominant answer since 2024 is to model the agent as an explicit state graph.

Graph-based state machines (LangGraph-style)

In LangGraph, you define a typed state object, nodes (functions, which may call LLMs or tools, that read state and return updates), and edges (fixed or conditional, where a function of state picks the next node). Updates are merged into state via per-key reducers (e.g. append to the message list rather than overwrite), which is what makes parallel branches safe to join. Execution proceeds in "super-steps", and with a checkpointer attached a state snapshot is saved at every super-step, keyed by a thread_id. LangGraph docs The same model covers fixed workflows (all edges fixed), ReAct agents (an LLM node ↔ tool node cycle with a conditional "tool calls?" edge), and multi-agent systems (nodes are subgraphs).

Checkpoints, human-in-the-loop and resumability

• Checkpoints give you: resume after a crash or deploy, "time travel" (inspect or fork from an earlier state to debug), multi-turn conversations (same thread_id), and HITL pauses that can last days. In production, use a persistent checkpointer such as Postgres.
• Interrupts: a node calls interrupt(payload), the graph saves state and returns, and later you resume with Command(resume=value), which becomes the return value of interrupt. One subtlety worth knowing: on resume, the whole node re-executes from its beginning, so side effects placed before the interrupt call run again and should be idempotent or moved after it or into a separate node. LangGraph docs
• HITL patterns: approve/reject a tool call; edit the tool arguments; edit the state (fix the plan); answer a clarifying question; review the final output. Put approvals on irreversible or high-blast-radius actions (payments, emails, deletes, prod deploys), not on everything, because approval fatigue turns humans into rubber stamps.

11. Durable execution for long-running agents

An agent that runs for minutes to hours, waits on humans for days, or calls flaky external APIs is a long-running distributed workflow, and it should be built like one. The lessons come straight from workflow engines like Temporal.

The Temporal model, applied to agents

Temporal splits code into workflows (deterministic orchestration code whose progress is recorded in an event history and recovered by replay after a crash) and activities (the non-deterministic, side-effecting work: network calls, DB writes, and, for agents, LLM calls and tool executions), which get automatic retries with configurable policies. Temporal docs Temporal docs Mapped onto an agent:

Why this matters: an LLM response is non-deterministic, so it must be an activity whose result is recorded. Otherwise replay would get a different answer and diverge. The workflow keeps only control logic. Temporal has published integrations with agent frameworks (for example a LangGraph plugin) that push LLM and tool calls into activities. Temporal blog

Idempotent tools, retries and sagas

• Idempotency keys: any side-effecting tool takes a key (e.g. hash(run_id, step_id)) so a retry after a timeout ("did the email send?") doesn't duplicate. Prefer upserts to inserts.
• Retry classes: retry transient errors (429, 5xx, timeouts) with exponential backoff and jitter. Don't blindly retry semantic errors (400, validation). Return those to the model as observations so it can fix its arguments.
• Sagas / compensation: for multi-step side effects across systems (book flight → book hotel → charge card), record a compensating action for each completed step (cancel flight, cancel hotel) and run them in reverse order if a later step fails. microservices.io For agents, declare compensations in the tool registry, because an LLM shouldn't improvise rollback.
• Timeouts at every level: per tool call, per LLM call, per step, per run. Hung tools are a top cause of stuck agents.
• Versioning: long-running runs outlive deploys. Workflow code changes need versioning so in-flight runs replay correctly, and prompts and tool schemas should be versioned too.

12. Concurrency and parallel tool calls

There are three levels of parallelism in agent systems:

1. Parallel tool calls within one turn. Modern model APIs let the model emit several tool calls in one response (e.g. three searches). The harness runs them concurrently (asyncio.gather) and returns all results together. It's cheap to add and cuts latency for independent lookups. In Anthropic's research system, parallel tool calls in subagents were one of the two parallelization levers behind the "up to 90%" time reduction. Anthropic 2025
2. Parallel branches in a workflow / graph. Fan out to nodes, fan in with reducers (LangGraph), or use a DAG executor (LLMCompiler).
3. Parallel agents. Subagents in separate contexts, possibly separate processes or sandboxes.

Back-of-envelope: a research step with 5 independent searches at about 2 s each takes about 10 s serially and about 2 s in parallel. Five subagents that each run 10 sequential steps of about 6 s (LLM + tool) finish in about 60 s wall-clock instead of about 300 s, at roughly the same or higher token cost. Parallelism buys latency, not cost.

13. Specialized agent types

Coding agents

Coding is the most commercially mature agent domain, because the environment gives dense, objective feedback (compilers, linters, tests) and the work is reversible (git). The typical loop: understand the task → explore the repo (search, read files) → plan → edit → run tests/linters → read failures → iterate → present the diff.

• Agent–computer interface (ACI). SWE-agent's key idea: LM agents are a new kind of user and need purpose-built interfaces. Its custom ACI had a file viewer showing a window of about 100 lines, search commands with concise output, and an editor that runs a linter and rejects syntactically invalid edits. That substantially improved performance over raw shell access, giving 12.5% pass@1 on SWE-bench at the time, which was state of the art. Yang+ 2024 The general lesson: tool design is prompt design. Anthropic reports spending more time optimizing tools than the overall prompt for its SWE-bench agent. Anthropic 2024
• Edit formats. There are several options: whole-file rewrite (simple, wasteful, risks truncation), search/replace blocks (exact-match old string → new string; the most common in production agents; fails loudly when the match isn't unique), unified diffs (compact but models make line-number errors), and AST- or LSP-aware edits. Requiring a unique exact match is a mistake-proofing trick: it forces the model to read before writing.
• Sandboxes. Run code in containers or microVMs (Docker, Firecracker-based services) with no prod credentials, restricted network egress, and resource limits. Permission tiers decide what's auto-allowed (read files, run tests), what needs approval (network, installing packages, git push), and what's denied.
• Test feedback. Tests are the verifier. Strong agents write a reproduction test first, then fix. Watch for agents that "make tests pass" by editing or deleting the tests, and guard against it with instructions and diff checks.
• Benchmarks: SWE-bench (real GitHub issues verified by hidden tests) Jimenez+ 2023 and its Verified subset are the standard references. Scores have risen fast, so quote current leaderboards, not memorized numbers.

Computer-use and browser agents

The action space is the GUI: screenshots (plus optionally the accessibility tree or DOM) come in as observations, and mouse/keyboard actions go out. Anthropic, OpenAI and Google all ship computer-use or browser-use model capabilities. Claude computer use docs Key engineering points:

• Grounding (clicking the right pixel/element) is a core failure source. DOM/accessibility-tree actions are more reliable than raw coordinates when they're available, so hybrid approaches are common.
• Latency and cost: each step sends a screenshot (roughly 1–2k image tokens), and tasks take tens of steps, so these agents are slow compared with API-based tools. Prefer an API or MCP tool whenever one exists and use GUI automation as the fallback.
• Security: web content is untrusted input, so prompt injection via page text is the main threat. Isolate in a VM/container, use allowlisted domains, keep credentials out of reach, and confirm sensitive actions with a human.
• Benchmarks: WebArena (realistic self-hosted websites) Zhou+ 2023 and OSWorld (real desktop OS tasks) Xie+ 2024.

Deep-research agents

The canonical orchestrator–worker use case: plan sub-questions → parallel search/read subagents → iterative deepening ("start wide, then narrow") → synthesis → citation verification. Design concerns: source quality ranking (prefer primary sources over SEO content), deduplication across subagents, claim-level citations checked against the fetched text, explicit "what I couldn't find" sections, and scaling effort to the question (don't spawn 10 agents for a factoid). Anthropic 2025 Evaluation is hard because answers are open-ended, so use LLM-judge rubrics (factual accuracy, citation accuracy, completeness, source quality, tool efficiency) plus human spot checks.

Voice agents

Voice agents run under a hard latency budget. In human conversation the gap between turns is short, and responses much slower than about ~1 s start to feel unnatural. That reshapes the architecture:

• Pipeline vs speech-to-speech: a cascaded STT → LLM → TTS pipeline (modular, any LLM, easy to inspect, more latency) vs native speech-to-speech realtime models (lower latency, better prosody, less control). OpenAI Realtime docs
• Latency budget (cascaded, rough): VAD/endpointing ~200–500 ms + STT finalization ~100–300 ms + LLM time-to-first-token ~200–500 ms + TTS time-to-first-audio ~100–300 ms. Everything streams, and the LLM's first sentence goes to TTS before the rest is generated.
• Turn-taking: endpointing (when has the user finished?), barge-in (the user interrupts, so stop TTS immediately and truncate the assistant message to what was actually spoken), and backchannels.
• Tools under latency: slow tool calls need filler speech ("let me check that…"), asynchronous tools, or prefetching. Keep agent graphs shallow, since multi-agent hops add round trips the caller hears as silence. Handoffs (not supervisors) are common in voice because they avoid an extra LLM hop per turn.
• Frameworks: LiveKit Agents and Pipecat are widely used open-source voice-agent frameworks. LiveKit docs Pipecat

14. Frameworks landscape (as of late 2026)

Framework vs roll your own

The common middle path: a thin in-house loop on the provider SDK, a real workflow engine for durability, MCP for tool integration, and OpenTelemetry-style tracing. Whichever you pick, make sure you can see the exact prompt sent to the model at every step. That's the debugging superpower frameworks sometimes hide.

15. Inter-agent protocols: MCP and A2A

16. Failure modes and mitigations

Why agents are hard, quantitatively: if each step succeeds independently with probability \(p\), an \(n\)-step task succeeds with probability \(p^n\). At \(p = 0.95\), 10 steps gives about 60% and 30 steps about 21%; at \(p = 0.99\), 30 steps gives about 74%. Real errors aren't independent and agents can recover, but the intuition holds: long horizons punish small per-step error rates, so verification and recovery matter more than any one step's accuracy.

17. How to choose an architecture

Work through these questions in order. Each "no" pushes you toward a simpler design.

Interview question bank