Agentic Workflow Patterns

Most production agents are made from a few well-understood patterns. This lesson covers them: chaining, routing, parallel work and maker-and-critic loops. It also covers planning, reflection and long-running execution. Each comes with its trade-offs.

Prompt chaining

Chaining breaks a task into smaller model calls. For example: pull out the requirements, classify them, make a plan and review the plan. The advantage is control. Each step has a smaller job and can be checked on its own. The disadvantages are more delay, more cost and errors that spread. Chain when the middle result is meaningful. If step two cannot be checked or reused, splitting may only add complexity. Always ask whether a step should be an LLM call at all. Fixed parsing, database queries, arithmetic and authorization are usually better in code.

Routing and classification

Routing sends an input down one of several specialized paths. Support requests might go to billing, technical support, account access or a person. It can be fixed rules, model-based or a mix. Use cheap rules first, and an LLM classifier only when the intent is fuzzy. Always define an unknown or needs-review path. Forcing every input into a category creates silent failures. Measure routing accuracy separately. One wrong route can make every part after it look broken.

Parallel work: fan out, fan in

Independent tasks run at the same time. For example, a research agent asks several sources at once, and then combines the answers. This cuts delay. But it increases concurrency, cost and failure handling. Every branch needs a timeout and a failure rule. Should one failed branch fail the whole task, be ignored, or trigger a fallback? Use a limited number of concurrent branches. Combine the results with clear rules for partial failure. Queues, workflow engines and asynchronous functions make this easier to run.

async def research(question, sources, per_branch_timeout=20, min_ok=2):
    sem = asyncio.Semaphore(4)                     # bounded concurrency
    async def one(src):
        async with sem:
            return await asyncio.wait_for(search_and_summarize(src, question), per_branch_timeout)

    results = await asyncio.gather(*(one(s) for s in sources), return_exceptions=True)
    ok = [r for r in results if not isinstance(r, Exception)]
    if len(ok) < min_ok:
        raise InsufficientEvidence(f"only {len(ok)} of {len(sources)} sources responded")
    return synthesize(question, ok, gaps=len(sources) - len(ok))   # tell the user what's missing

Maker and critic, reflection and verification

A generator produces something. A critic checks it against criteria. The system may then revise it. This can improve writing, code, extraction and plans. The danger is an expensive self-criticism loop that only writes confident prose about its own mistakes. Use objective signals wherever you can. These are schema checks, tests, retrieved evidence, business rules and reference answers. For code agents, run the tests. For extraction, compare with the known fields. For policy answers, check the citations. LLM judges can help, but judge prompts and models need checking too. Limit the rounds of refining. Stop when extra rounds buy little measurable improvement.

Reflection asks the model to look at its own result. Verification uses independent evidence, or a separate process, and it is stronger. For arithmetic, calculate in code. For search, check that the answer matches the sources. For actions, look at the state of the system afterwards. Reflection is most useful when the criteria are clear and the fix step can act on them. Use it selectively, because of its cost and delay.

Planning and long-running tasks

Planning makes a sequence of actions toward a goal. The plan can be text, a structured graph of tasks or state-machine steps. It helps when tasks have dependencies. But a generated plan is not automatically correct. Check it against the available tools, permissions, budgets and prerequisites. A mix often works best. Code sets the hard limits and the allowed transitions, and the model chooses among the permitted options.

Long-running tasks need lasting state, checkpoints, asynchronous events, heartbeats, timeouts, cancellation and recovery. A user-facing API should return a run ID quickly, while the backend works in the background and the interface polls or subscribes. Each step records its progress and its results. Cancellation is a first-class feature. A user must be able to stop an agent before a destructive action that is pending. And tools must check the cancellation state before they go ahead.