LLM Agent Architecture: Tool Design, Control Loops, State, and Failure Handling

The loop, and its bounds

async function run(task: string, limits: Limits) {
  const messages: Message[] = [{ role: "user", content: task }];
  let spent = 0;

  for (let step = 0; step < limits.maxSteps; step++) {
    const res = await client.messages.create({ model, tools, messages });
    messages.push({ role: "assistant", content: res.content });
    spent += cost(res.usage);

    if (spent > limits.maxCostUsd) return halt("budget", messages);
    if (res.stop_reason !== "tool_use") return done(res, messages);

    const results = await Promise.all(
      toolCalls(res).map((call) => execute(call, { timeoutMs: 30_000 }))
    );
    messages.push({ role: "user", content: results });

    if (tokensOf(messages) > limits.compactAt) await compact(messages);
  }
  return halt("step_limit", messages);
}

Every bound in that function exists because of a specific production incident somewhere. Step limits stop loops where the model retries a failing tool forever. Cost ceilings stop a runaway from becoming an invoice. Per-tool timeouts stop one hung HTTP call from consuming the entire run. Add them before launch, not after.

Tool design is API design for a reader who cannot ask questions

The model sees only your tool names, descriptions, and parameter schemas. It cannot read your source, ask a colleague, or check a wiki. Treat each description as documentation for a competent new engineer with no context — including what the tool does not do, and what to do when it fails.

• Few, capable tools beat many granular ones. Ten tools that each do a real unit of work outperform forty that must be composed in exactly the right order.
• Name by intent, not implementation: search_customers, not query_pg_users_table.
• Constrain in the schema — enums, formats, ranges. A constraint in the schema is enforced; the same constraint in prose is a suggestion.
• Return errors the model can act on. 'Customer not found. Search by email with search_customers first' recovers; 'Error: 500' produces a retry loop.
• Return structured, compact results. Dumping a 40KB JSON blob into context burns budget and buries the answer — paginate and summarize at the tool boundary.
• Make read tools obviously safe and write tools obviously consequential, and say so in the description.

If you are exposing the same capabilities to more than one agent or client, define them once behind MCP rather than reimplementing the schema per integration. The tool contract becomes a server you version, test, and reuse.

Context over long runs

Agents fail slowly. Twenty tool calls in, context is full of stale intermediate output, and the model is reasoning over noise. Manage this explicitly rather than hoping the window is large enough.

• Compact at a threshold: summarize completed subtasks into a short record of what was learned and what remains, and drop the raw transcript behind it.
• Keep an explicit external state object — the task, its decomposition, findings so far, open questions — and re-inject it after compaction. This is the agent's memory of intent, and it must survive.
• Write large artifacts to files or a store and pass identifiers, not contents.
• Cache the stable prefix. System prompt and tool definitions are identical across every step of a run; cached, they cost a fraction of the tokens.

Single agent or many?

Multi-agent architectures are frequently reached for too early. Every additional agent adds a lossy handoff, and coordination overhead grows faster than the parallelism gains. The honest default is one agent with good tools.

When you do fan out, give each subagent a self-contained brief. A subagent starts cold: it cannot see the parent's reasoning, so anything it needs must be written into its prompt. Vague briefs are the main reason multi-agent systems underperform the single-agent baseline they replaced.

Where humans belong

Agents should be trusted with reversible actions and gated on irreversible ones. That line — not model quality — is what makes an agentic system deployable inside a company that has auditors.

• Autonomous: reading, searching, drafting, analysis, anything in a sandbox, anything that can be undone with one command.
• Confirmed: sending communications on someone's behalf, moving money, deleting data, changing production config, publishing.
• Logged regardless: every tool call with arguments, result, latency, and cost, tied to a trace ID a human can replay end to end.
• Idempotent by construction: every write tool takes a client-supplied key, so a retried step cannot double-charge, double-send, or double-post.