Guardrails & Agent Security

A chatbot that says something wrong is embarrassing. An agent that does something wrong is an incident. It might email a customer list to a stranger, delete a live table or move money. Agents read text and act on it. So anyone who can affect that text can try to affect the actions. Securing agents is mostly about limiting what a tricked agent could do, instead of hoping it cannot be tricked.

The kinds of threat

• Direct prompt injection: the user tells the agent to ignore its rules.
• Indirect prompt injection: instructions are hidden in something the agent reads, like a web page, a PDF, an email, a calendar invite, a tool result or another agent's message. This is the dangerous one, because the attacker is never a user.
• Data theft: tricking the agent into sending private data out, for example in an email, in a web address it opens, or in an image link that carries the data.
• Too much power: the agent has more tools, permissions or freedom than its task needs, so any mistake or trick can do a lot of damage.
• Misused tools: the agent uses its own legitimate logins for an attacker.
• Poisoned memory or documents: harmful text is stored and later found and trusted as context.
• Supply chain: a hacked MCP server, plugin, tool description or software dependency.

The OWASP Top 10 for LLM Applications is a good checklist to follow. Prompt injection and too much power both appear in it. Security researcher Simon Willison has a framing that is very useful for agents.

The lethal trifecta

Suppose one agent has all three of these: (1) access to your private data, (2) exposure to untrusted content and (3) a way to send data out. Then an attacker who can plant instructions in the untrusted content can make the agent steal the private data. You do not need a clever filter. You need to remove one of the three. This is the idea of splitting by permission from the orchestration path, now stated as a security rule.

For every agent, answer YES/NO:
  [ ] Private data:      can it read anything non-public (CRM, email, files, DB)?
  [ ] Untrusted content: does it read content authored by outsiders (web, inbound mail, tickets, uploads)?
  [ ] External channel:  can it send data out (email, HTTP requests, links, write to shared docs, rendered images)?

3 x YES  -> redesign. Split into two agents, remove a leg, or put a human approval on the external channel.
2 x YES  -> document the risk and add compensating controls.

Defenses that work, most reliable first

• Least privilege: give each agent only the tools and scopes its job needs. Read-only by default. Separate logins for each agent. Short-lived tokens.
• Separation by design: keep the agent that handles untrusted content apart from the one that can act. Pass only checked, extracted data between them.
• Rules enforced outside the model: a code layer that checks every tool call against rules, such as allowed tools, allowed domains and amount limits, whatever the model decided.
• A person approves high-impact actions (the next lesson).
• Sandboxing: run generated code and browsing in isolated containers with no secrets, a limited file system and an allow-list for network access.
• Filters and classifiers: useful as a speed bump and for monitoring, but never your only defense. A determined attacker gets past them.
• Safe display: do not auto-load images or links from model output. Clean any markdown or HTML, to close the data-theft routes.
• Secret hygiene: never put logins in prompts. Add them in the tool layer, so the model never sees them.

ALLOWED = {
  "crm.read_contact":   {"max_per_run": 20},
  "email.send":         {"allowed_domains": ["ourcompany.com"], "requires_approval": True},
  "refund.issue":       {"max_amount_usd": 50, "requires_approval_above": 20},
}

def enforce(tool: str, args: dict, run) -> None:
    rule = ALLOWED.get(tool)
    if rule is None:
        raise PolicyDenied(f"{tool} not permitted for this agent")
    if tool == "email.send" and args["to"].split("@")[-1] not in rule["allowed_domains"]:
        raise PolicyDenied("external recipient")
    if tool == "refund.issue" and args["amount_usd"] > rule["max_amount_usd"]:
        raise PolicyDenied("amount over hard limit")
    if run.count(tool) >= rule.get("max_per_run", 10**9):
        raise PolicyDenied("rate limit for this run")
    # approvals are handled by the caller when 'requires_approval*' applies

# Call enforce() inside the tool dispatcher - the model cannot talk its way past code.

MCP and the tool supply chain

• Check MCP servers like any outside software: the source, the maintainers, the permissions they ask for and how they update.
• Pin versions and review changes. A tool's description is text the model reads, so a bad update can change how the agent behaves.
• Give each server its own login scope, and prefer read-only tokens.
• Watch out for tool-name clashes and look-alike servers.

Test like an attacker

Build a test suite of attacks. Plant instructions in documents, emails and tool results. Ask it to reveal the system prompt or secrets. Try to steal data through links and images. Try to go past the tool limits. Run the suite automatically and on a schedule. Add every real attack that works. Every so often, run a session where people try to break it. Track the attack success rate as a number.