Failure Modes & Reliability

Teams of agents fail in ways a single agent does not. Researchers who studied hundreds of real runs from popular multi-agent frameworks found that failures fell into three families. The first is problems in how the task and the roles were set out. The second is problems in how the agents worked together. The third is problems in checking the results and knowing when to stop. None of these is about how smart the model is. They are design problems. That is good news, because design problems have design fixes.

A catalogue of failures

• Endless loops and ping-pong: agent A asks B, B asks A, forever. Or a critic and a writer never agree.
• Stopping too early: an agent says it succeeded because it ran out of ideas, not because the goal was met.
• Spreading inventions: one agent makes up a fact, the next treats it as true, and the final answer states it with confidence.
• Role confusion and ignored input: an agent drifts out of its role, or ignores a correction from a peer.
• Lost context at hand-offs: the receiver does not know a limit that the sender knew.
• Repeated or clashing work: two workers do the same thing, or undo each other's changes.
• Runaway spawning: a lead starts workers that start more workers, and the bill explodes.
• Fighting over a tool: parallel agents hit the same service and trigger rate limits or lock clashes.
• Quiet partial failure: a worker fails, and the system reports success using the rest of the results.

Guardrail 1: hard budgets on everything

Every loop, every fan-out and every run needs a number limit that no prompt can argue with. Budgets are the most valuable reliability feature you can add. They turn failures with no end into failures with a limit, which you can see.

from dataclasses import dataclass, field
import time

@dataclass
class Budget:
    max_steps: int = 40
    max_tokens: int = 400_000
    max_seconds: float = 300
    max_spawn_depth: int = 2
    max_workers: int = 8
    steps: int = 0
    tokens: int = 0
    started: float = field(default_factory=time.monotonic)

    def charge(self, tokens: int = 0):
        self.steps += 1
        self.tokens += tokens
        if self.steps > self.max_steps:           raise BudgetExceeded("steps")
        if self.tokens > self.max_tokens:         raise BudgetExceeded("tokens")
        if time.monotonic() - self.started > self.max_seconds:
                                                  raise BudgetExceeded("time")

# Pass the SAME Budget object through the whole run, including into sub-agents.
# On BudgetExceeded: stop, return what you have, flag it as partial.

Guardrail 2: clear ways to stop

Tell each agent what done means in terms you can check. For example: fields filled in, tests passing, or N sources cited. Have a checker outside the agent that made the work confirm it. Finish when you are satisfied invites both stopping too early and polishing forever. Where you can, make the final check plain code: run the tests, check the schema, and check that every claim has a source address.

Guardrail 3: check at the boundaries

• Check every hand-off and result against its schema. Reject and retry the ones that are malformed.
• Ask for references or evidence IDs for factual claims. Spot-check that the cited source really says it.
• Use an independent checker for high-stakes claims, one that does not see how the maker reasoned.
• Label unchecked facts as unchecked when they cross a boundary, so the agents after them can weigh them.

Guardrail 4: the habits of distributed systems

• A timeout on every outside call and every worker.
• Retries with longer waits and a little randomness, only for errors that are really worth retrying.
• Idempotency keys on side effects, so retries and resumes cannot do the same thing twice.
• Circuit breakers: after N failures in a row from a service, stop calling it for a while and carry on in a reduced way.
• Dead-letter handling: work that keeps failing goes to a queue that a person reviews, and does not loop.
• Graceful degradation: return a partial result with an honest list of gaps, and do not fail the whole run.

Partial results are a feature

Say one of eight workers fails. The right behavior is usually to deliver the seven good results, with a clear note of what is missing and why. Do not retry forever, and do not pretend nothing happened. Design your combining step to accept and report gaps. And make incomplete a proper status for a run.