Memory, State & Multi-Agent Systems
Short-term state, long-term memory with provenance, and when multiple agents are genuinely justified.
Short-term memory is what the AI needs for the current job. Long-term memory is saved facts, so note where each came from and when, and let users fix or delete it. Use several agents only when each needs different information or permissions.
- Provenance
- Where a piece of information came from.
- Supervisor
- The agent that hands out work and keeps control.
- Retention
- How long data is kept.
Use several agents only when each needs different information or permissions.
- What the AI needs for the current job
- Cleared when the job ends
- Saved facts
- Note where each came from and when
- Users can fix or delete it
An agent that cannot remember is limited. An agent that remembers badly is dangerous. This lesson treats memory as data management, with sources, retention and access rules. Then it looks at when using several agents is really justified.
Short-term memory and conversation state
Short-term memory is the state needed to carry on the current task. It holds recent messages, tool outputs, selected documents and goals. Do not blindly add every message forever. Context and cost grow, and irrelevant history can lower the quality of answers. Use summaries, selective keeping, structured state and retrieval of earlier events. A good state model separates the conversation the user sees from the machine state. Machine state is IDs, statuses, retries and internal details, and it should not be shown to the model or the user.
- Store structured state apart from the raw conversation text.
- Summarize or retrieve history. Do not let the context grow without limit.
- Keep secrets and authorization state out of memory the model can see, unless it is truly needed.
Long-term memory
Long-term memory lasts beyond one task or session. It holds user preferences, earlier decisions, company policies and learned facts. Every memory should carry its source, a timestamp, a confidence level, its scope and rules for deleting it. A memory that was true six months ago may be false now. And a memory without a source becomes an untraceable cause of made-up answers. Define clear policies. What may be stored? Who can access it? How long does it stay? How do users correct or delete it? In company systems, separating customers and meeting compliance rules are critical.
{
"id": "mem_77",
"tenant_id": "acme",
"subject": "user:u_1042",
"kind": "preference",
"text": "Prefers invoices summarised in a table",
"provenance": { "run_id": "run_8f3a", "source": "user_message", "quote": "put it in a table please" },
"confidence": 0.9,
"created_at": "2026-09-12T10:04:00Z",
"expires_at": "2027-03-12T10:04:00Z",
"visibility": "user_and_agent",
"user_can_edit": true
}
// Memory is data management, not magic: store provenance and confidence, define retention and correction policies BEFORE you deploy persistent memory.Multi-agent systems
A multi-agent system uses several specialized agents or roles, such as planner, researcher, coder, reviewer and executor. It can improve modularity and parallel work. But it also raises the cost of coordination, delay, ways to fail and token use. Many so-called multi-agent systems can be simplified into one agent plus fixed tools. Use several agents when the responsibilities truly need different contexts, permissions, expertise or running environments. And keep the communication contract clear.
- A strong design often has one supervisor in control of delegation and policy, while specialized workers do bounded tasks. Avoid uncontrolled chatter between agents.
- Limit how deep delegation goes, and the total work in steps, tokens, time and workers.
- Log every hand-off with a task ID and a correlation ID, so that a run can be rebuilt.
- Use specialization only where it creates a measurable advantage over the single-agent baseline.
- ✓Keep machine state apart from conversation text. Summarize or retrieve instead of adding forever.
- ✓Long-term memory needs a source, confidence, scope, retention and a way for users to correct or delete it.
- ✓Multi-agent designs cost more in delay, tokens and ways to fail. Justify each agent with different context, permissions or environments.
- ✓Use a supervisor plus bounded workers, with correlation IDs on every hand-off.
Design a memory policy, and defend your number of agents
- Write the memory policy for a support assistant. What may be stored, what must never be stored, the retention periods, who can view and edit, and how deletion reaches indexes and caches.
- Build a store with sources and expiry. Show an old memory being replaced by a newer fact.
- Take a multi-agent diagram, yours or an example, and try to collapse it into one agent plus tools. List what you cannot collapse, and why.
- Add correlation IDs across a hand-off between two agents, and rebuild one run from the logs.
Q1Why store the source with each memory?show answer
Without knowing where a fact came from and when, you cannot check, correct or expire it. It becomes an untraceable cause of made-up answers.
Q2When are multiple agents justified?show answer
When the roles truly need different contexts, permissions, expertise or running environments, and a measured benefit over a single agent outweighs the cost of coordination.
Q3What should not be in memory the model can see?show answer
Secrets and authorization state. The system enforces those. The model should not remember them.