The Seven Orchestration Patterns

Almost every successful team of agents is a mix of a few patterns that come up again and again. Learning them by name matters. It lets you design from a shared vocabulary instead of from nothing, and it lets you think about how each one fails. This lesson covers the seven you will use over and over.

1. Prompt chain (pipeline)

The output of step A feeds step B, which feeds step C. Each step is a separate model call with its own focused prompt. You can put a plain code check between steps. Use it when a task splits cleanly into fixed stages in order, like extract, classify, draft and format. It is the cheapest and most predictable pattern, and it is often the right answer. The pitfall: errors flow downstream. So add cheap checks between stages, such as schema checks, length checks or a quick yes or no from a model.

2. Router

A classifier step decides which specialist or which model handles the request. Use it when the inputs fall into clear groups that need different prompts, tools or model sizes. The pitfall: the router is a single point of failure. A wrong choice quietly sends work to the wrong specialist. Log every routing decision with its confidence. Keep a none-of-the-above lane that goes to a generalist or a person.

ROUTES = {
    "billing":  billing_agent,    # has: account_lookup, refund_tool
    "technical": tech_agent,      # has: docs_search, log_viewer
    "other":    generalist_agent, # safe default, escalates freely
}

def route(message: str) -> str:
    # cheap, fast model; constrained output
    result = small_model.generate(
        system="Classify the support message. Reply with JSON only.",
        user=message,
        schema={"route": ["billing", "technical", "other"], "confidence": "0..1"},
    )
    if result["confidence"] < 0.7:
        return "other"           # low confidence -> generalist, never guess
    return result["route"]

def handle(message: str):
    chosen = route(message)
    log("routed", chosen=chosen)          # you will want this in traces
    return ROUTES[chosen].run(message)

3. Parallel work: sections and votes

Sectioning splits a task into independent parts that run at the same time. For example, summarize 20 documents, or review each file in a pull request. Voting runs the same task several times and combines the answers. For example, three reviewers look for a security flaw, and you take all findings, or require two of three. Use sectioning for speed and breadth. Use voting for confidence on high-stakes decisions. The pitfall: parts that are not truly independent will contradict each other when you merge them. Decide how you will merge before you split.

4. Orchestrator and workers

A lead agent makes a plan while it runs, starts workers with the sub-tasks it chose, and combines their results. This differs from sectioning because the sub-tasks are not known in advance. The lead decides from the input. It is the pattern behind most deep-research products. The pitfalls are a lead that gives vague instructions, so workers repeat each other or wander off, and a lead that starts too many workers. Clear written briefs and effort limits fix both. The next lesson covers them.

async def research(question: str) -> str:
    plan = await lead.plan(question)            # -> list[SubTask], decided at run time
    assert len(plan) <= MAX_WORKERS             # hard cap, not a suggestion

    results = await asyncio.gather(
        *(worker.run(task) for task in plan),   # parallel fan-out
        return_exceptions=True,
    )
    good = [r for r in results if not isinstance(r, Exception)]
    return await lead.synthesize(question, good)  # lead owns the final answer

5. Evaluator and optimizer (maker and critic)

One agent makes something. Another judges it against clear criteria. The loop repeats until the critic passes it or a round limit is reached. Use it when you can state clear quality rules and going round again really helps. Examples are translation, code that must pass tests, or text that must meet a checklist. The pitfall: a model critic with vague rules either approves everything or nitpicks forever. Give it a checklist. Prefer code checks, such as tests, linters and schema checks, wherever they exist. And always cap the rounds.

6. Handoff (no manager)

Instead of a manager, agents pass control to each other. The triage agent hands the conversation to billing, who may hand it to a person. The OpenAI Agents SDK made this popular under the name handoffs. Use it for conversations where the user talks to whichever specialist is in charge right now. The pitfall: nobody owns the whole result. Loops (A to B to A) and lost context are common. Pass the conversation state clearly, and do not allow an agent to hand straight back.

7. Hierarchy (supervisors of supervisors)

When one lead would have too many workers or too much to track, you can nest them. A top lead gives work to team leads, who give work to workers. It scales coordination the way management scales a company, and it adds the same overhead. Use it only after a single flat lead clearly cannot cope, and keep it to two levels where you can.

Combining patterns

Real systems layer these patterns. A research product might work like this. A router asks: is this a quick lookup or deep research? Then a lead with parallel searchers does the work. A maker-and-critic loop checks the draft for facts. A simple chain handles the formatting and the sources. Each layer should be testable on its own and easy to remove. If you cannot switch a layer off and compare, you cannot prove it helps.