System Design Case Studies
A repeatable design method applied to an enterprise assistant, a support agent, a coding-agent platform and an agent platform.
System design is planning how a system fits together. Follow the same steps each time: understand the goal, start with a simple version, draw how data flows, list what can fail, think about safety, then add parts only when needed. Try it on four real examples.
- Baseline
- The simplest version that could work.
- Control plane
- The part that manages settings and rules.
- Data plane
- The part that does the real work.
System-design interviews and real architecture reviews use the same method. Start from the business outcome. State the requirements and the risks. Draw the smallest design that works. Then add a part only when a requirement demands it. This lesson applies the method to four systems.
The method
The steps below work for any system. Do them in order. Say your assumptions out loud. Skipping the early steps is the most common reason a design goes wrong.
1. Requirements users, tasks, data sources, actions, latency (p95), availability, privacy, compliance, failure impact 2. Constraints budget per task, data retention, audit needs, regions, team skills 3. Baseline the simplest design that could work (often: UI -> API -> auth -> retrieval -> model -> citations) 4. Data flow draw it; mark trust boundaries and state boundaries 5. Failure modes what breaks, how you detect it, how you recover 6. Security identity, authorization, injection, exfiltration, tenant isolation 7. Observability traces, metrics, evals, alerts 8. Scale and cost load, token budget, caching, routing 9. Alternatives what you rejected and why Add tools only when users need actions; add agent loops only when the task path is dynamic.
Case 1: an enterprise AI assistant
Start with requirements. Who are the users? What tasks do they do? Which data sources and actions are involved? How fast must it be? How available? What are the privacy and compliance rules? What happens when it fails?
For a knowledge assistant, the baseline is simple. The user interface calls the API. The API checks who the user is. Then it retrieves documents, calls the model and returns an answer with citations. Add tools only if users need actions. Add agent loops only if the path through the task changes each time.
Write the non-functional requirements as numbers. Examples are p95 delay, target groundedness, the maximum cost per task, how long you keep data, audit needs and availability. These drive the architecture far more than the choice of framework. Keep knowledge access apart from action execution. And define the service goals and risk controls before you pick a framework.
Case 2: an AI customer support agent
A support agent needs customer context, knowledge search, ticket APIs, policy checks and a way to escalate. It must not make unrestricted changes. Take a refund tool as an example. It needs a maximum amount. It needs customer verification and eligibility rules. Above a set amount it needs human approval.
Log the evidence behind each answer and each action taken. Test with realistic conversations. Also test with adversarial attempts to reach another customer's data. You are checking two things: answer quality and action correctness.
- Keep read tools apart from write tools.
- Add clear paths for escalation to a person.
- Evaluate answer quality and action correctness together.
Case 3: a coding agent platform
A coding agent needs isolation. Run it in a sandbox. Give it only the repository and the tools it needs. Restrict the network. Record every command and every file change. Run tests before it says the work is complete.
In a company, connect it to the normal engineering process. That means branches, code review, CI, static analysis, dependency scanning and approval gates. The agent must leave a work trail that someone else can reproduce. Three properties matter most: sandboxed execution, an audit trail that cannot be changed, and automatic checks before a person reviews.
Case 4: an internal agent platform
An internal platform gathers shared needs in one place. These are model access, tool registration, identity, policies, evaluation, tracing, memory and deployment standards. Applications use these abilities. Each team does not rebuild login, model routing, safety and telemetry.
Offer SDKs and templates. But do not force every workload into one style of orchestration. A strong platform splits into two parts. The control plane holds configuration, policies, tool details, models, evaluation suites and tenants. The data plane runs the real user workloads.
Applications / UIs
-> API + Identity + Tenant context
-> Agent runtime / workflow engine
|-> Models (via model gateway)
|-> Retrieval (vector + SQL, tenant-filtered)
|-> Tools (API / Lambda, registry-governed)
-> Policy + guardrails (deterministic)
-> State + event bus
-> Observability / evaluation / audit
Control plane: models, policies, tool metadata, eval suites, tenants. Data plane: user workloads.- ✓Design from business outcomes. Cover requirements, constraints, a baseline, data flow, failure modes, security, observability, scale and alternatives.
- ✓Non-functional requirements drive the architecture. Write them as numbers: p95, groundedness, cost per task, retention and audit.
- ✓Keep knowledge access apart from action execution, and read tools apart from write tools.
- ✓Platforms win by centralizing shared concerns while keeping execution isolated and able to scale.
Run the method on a system you pick
- Choose one of the four cases, or your own. Write the requirements and the non-functional targets as numbers.
- Draw the baseline design. Then list each part you added and the requirement that justified it.
- List the top five failure modes and the security risks. Add a preventive control and a detective control for each.
- Present it in five minutes to a colleague. Then write down every challenge they raised and your answer.
Q1Why define service goals and risk controls before choosing a framework?show answer
They decide the architecture: delay, groundedness, cost, retention and audit. A framework is an implementation detail. You choose one that fits those needs.
Q2In a support agent, what protects against refund abuse?show answer
Fixed policy in the refund tool. It sets limits, verifies the customer, checks eligibility and requires human approval above a threshold. All actions and evidence are logged.
Q3What is the difference between a control plane and a data plane in an agent platform?show answer
The control plane manages configuration, policies, tools, models, evals and tenants. The data plane runs the user workloads in isolated runtimes that can scale.