What Agent Ops Is (and Why It's Different)

Getting an agent to work in a demo takes an afternoon. Keeping it working for thousands of real users is a different job. You have to get through model updates, strange inputs, hostile inputs, outages and growing bills. That discipline is agent operations. It is the set of practices that keep an agent safe, visible, affordable and improving once it is live. It sits next to classic DevOps and MLOps, but it is not the same as either. This path teaches it from start to finish.

Why agents need their own ops discipline

• They are not predictable. The same input can take a different path next time, so it passed once proves little. Quality is a spread of results, not a yes or no.
• They act. Agents call tools that send emails, change records and spend money. A bug is not just a wrong answer. It can be a wrong action.
• They run long and in many steps. Failures hide in the middle of a 40-step run, far from the symptom.
• They depend on things that move. Model providers update, retire and limit models, so your behavior can change without you touching a line of code.
• They can be attacked through language. Anyone who can put text in front of the agent, in an email, a web page or a support ticket, is a possible attacker.
• They cost money on every request. Cost grows with tokens and turns, so a bug in a loop is a billing bug.

The eight pillars

Think of agent ops as eight pillars. Each gets a lesson in this path. A launch review, in the last lesson, scores you on all of them.

• Observability: you can see exactly what the agent did and why, for any run.
• Evaluation: you can measure quality before and after every change.
• Guardrails and security: the agent cannot be talked into causing harm.
• Human oversight: the right actions get a person's approval, and leave a record.
• Cost and usage: you know and control what each run, user and customer costs.
• Reliability: failures reduce service gently instead of spreading.
• Change management: prompts, models and tools are released like code, with checks and a way back.
• Governance: ownership, policy, compliance and incident response are set up before you need them.

Say what success means with SLIs and SLOs

Site reliability engineering gives us the words. A service-level indicator (SLI) is something you measure. A service-level objective (SLO) is the target for it. Agents need quality and safety indicators, as well as the usual speed and error rate.

agent: support-triage
slis_and_slos:
  task_success_rate:        { target: ">= 88%", window: "7d", measured_by: "outcome eval + resolved-without-reopen" }
  escalation_rate:          { target: "8-15%",  note: "too low = overconfident, too high = useless" }
  p95_latency_seconds:      { target: "<= 25" }
  cost_per_successful_task: { target: "<= $0.12" }
  tool_error_rate:          { target: "<= 2%" }
  guardrail_block_rate:     { target: "tracked, alert on 3x change" }
  harmful_action_incidents: { target: "0", severity: "page someone" }
  user_satisfaction:        { target: ">= 4.2/5", sample: "thumbs + periodic survey" }

A maturity ladder

• Level 0, hope: the agent runs, the logs are print statements, and you hear about problems from users.
• Level 1, visible: traces, cost and error dashboards exist, and someone looks at them.
• Level 2, measured: a set of regression tests gates changes. Targets and alerts exist. Risky actions need approval.
• Level 3, governed: ownership, incident runbooks, audit logs, staged releases, off switches and regular red-teaming are routine.

Most teams reach Level 1 by accident and stay there. Moving to Level 2 is the step that gives the most. It is where you stop shipping on gut feeling. The rest of this path is the map.