Multi-Step Planning, Memory & State

Many companies are adding agents to their software. Nearly all of them need an agent to stay on track for a task that is longer than one context window. This lesson gives the real answer to the problem the first lesson only pointed at: what do you do when the history will not fit?

Three kinds of memory

• Episodic: what happened in this session or task. The turns, the tool calls and the results so far.
• Semantic: facts and knowledge. Things that are true whichever session you are in.
• Procedural: how things should be done. Workflows, tool habits, house rules and deployment steps.

Most simple agents have only episodic memory, and only for the current session. That is why they repeat mistakes from one session to the next. A procedural memory would already have learned from them.

Two tiers: in the context and outside it

Letta (it used to be called MemGPT) keeps memory in two tiers. In-context memory is what is in the model's window right now. Out-of-context memory is everything else. It sits in outside storage and is fetched when needed. This turns the context budget problem from the first lesson into a design. Instead of one growing list of every message ever sent, you keep a small working set in view. You treat the rest as something you look up, the same way RAG treats a pile of documents.

Memory chunked by subgoal

The HiAgent pattern manages working memory in levels. Instead of one flat, ever-growing history, the agent splits a long task into subgoals. The history of each subgoal is its own chunk. When a subgoal is done, its detailed history can be shrunk to a single result, and the full detail can be dropped. That gives real savings in context. It is not just better planning inside a window that keeps growing.

type Subgoal = { goal: string; messages: Message[]; result?: string };

function compressCompletedSubgoal(sg: Subgoal): Message {
  // once a subgoal is done, collapse its detailed history to one summary
  // message instead of carrying every intermediate tool call forward
  return { role: "assistant", content: `[${sg.goal}] -> ${sg.result}` };
}

Going deeper: planning patterns

• ReAct (reason, then act): the model takes turns thinking and calling tools. It is simple and adapts well. It is weaker on long tasks because it can wander.
• Plan and execute: write a plan first, then do it step by step, and make a new plan if a step fails. This is better for multi-step goals, and easy to show to a person for approval.
• Reflection: after it writes something, the model, or another model, criticizes it against set criteria and fixes it. It helps when the criteria can be checked. Limit the rounds.
• Search over options: make several candidate plans or answers and pick the best with a checker. It costs more. Use it for high-value decisions.

def solve(goal: str, budget: Budget):
    plan = planner.make_plan(goal)                         # list[Step], validated schema
    results = []
    for step in plan:
        budget.charge()
        out = executor.run(step, context=compact(results)) # only what this step needs
        if not verifier.ok(step, out):
            plan = planner.replan(goal, done=results, failed=(step, out))   # bounded replans
            if budget.replans_exceeded(): return partial(results, reason="could not complete")
            continue
        results.append((step, out))
    return synthesize(goal, results)

Going deeper: building a memory store

Long-term memory is a small database with a rule for writing and a rule for finding. If you get the rules right, a modest store goes a long way. If you get them wrong, the agent will confidently recall things that were never true or are no longer true.

# memory(id, user_id, kind, text, embedding, source_run, confidence, created_at, expires_at)

def maybe_remember(user_id, candidate, run):
    if not is_durable(candidate): return            # one-off details are not memories
    if contains_secret_or_pii(candidate): return    # never store credentials or sensitive data
    if not candidate.verified and candidate.confidence < 0.8: return
    existing = nearest(user_id, candidate.text, threshold=0.92)
    if existing: update(existing, candidate)        # dedupe / supersede, don't pile up
    else: insert(user_id, candidate, expires_at=ttl_for(candidate.kind))

def recall(user_id, query, k=5):
    hits = vector_search(user_id, query, k=k * 3, filter={"expires_at": ">now"})
    return rerank(hits, by=("relevance", "recency", "confidence"))[:k]   # and show it to the user

• Writing rule: store only lasting, useful, checked facts and preferences. Record where each one came from.
• Keep memory separate for each user or customer, and filter BEFORE you search. A leak of memory between users is a serious security bug.
• Give memories an expiry time, and let newer facts replace older ones.
• Let users see, edit and delete what the agent remembers about them.
• Test memory. Check that recalled facts help on follow-up tasks, and that old facts get corrected.