The word "agent" hides a lot of engineering. Strip away the hype and an agent is a loop: the model looks at a state, chooses a tool, observes a result, and repeats until it decides it is done. Every one of those steps is a place where things go wrong.
The three failures we design against
Every unreliable agent we have debugged failed in one of three ways.
- The runaway loop — it never decides it is done and keeps calling tools.
- The confident wrong turn — it calls the right tool with the wrong arguments and never notices.
- The irreversible action — it does something it cannot undo before a human can intervene.
Prompts do not fix these. Architecture does.
Bound the loop
Give the loop a hard budget — steps, tokens, and wall-clock — and make hitting it a graceful outcome, not a crash:
def run_agent(task: str, max_steps: int = 8) -> Result:
state = init(task)
for step in range(max_steps):
action = model.decide(state)
if action.is_final:
return Result(answer=action.answer, steps=step)
state = apply(state, tool_call(action))
return Result(answer=None, steps=max_steps, reason="budget_exhausted")An agent that gives up cleanly is infinitely more useful than one that loops forever.
Separate reversible from irreversible
Classify every tool. Reversible tools (search, read, compute) run freely. Irreversible tools (send email, charge a card, delete a record) pass through a confirmation gate — human-in-the-loop, or at minimum a policy check — before they execute. This single distinction prevents the scariest class of agent failure.
Make it observable
You cannot trust what you cannot see. Log every decision as a structured trace — the state, the chosen action, the arguments, the result — so that when an agent does something surprising you can replay exactly why. Traces are not a nice-to-have; they are the difference between a system you operate and a system you pray to.
Reliability is the product
Anyone can wire a model to a set of tools. The engineering — the budgets, the gates, the traces, the graceful failure — is what makes an agent something you would put in front of a customer. That engineering is the product.