Agent & Tool Security

A chatbot that only talks is a limited risk. An agent that can act is a larger one. It can send email, change records, run code and spend money. If an attacker steers it, the attacker borrows its power. This lesson covers how to give agents only the power they need, and how to keep humans in control of risky steps.

Excessive agency

Excessive agency means an agent has more tools, more permissions or more freedom than its task needs. It is a top risk because the damage of any other attack grows with it. A tricked agent with read-only access can leak. A tricked agent with admin access can destroy.

Least privilege for agents

• Give each agent only the tools it needs for its job.
• Give each tool the narrowest permission: read-only where possible, one table or folder, short-lived keys.
• Run actions as the end user, so the agent can never do more than the user may.
• Use separate identities for separate agents, so one stolen key does not open everything.
• Prefer many narrow tools over one powerful tool, such as a tool that can run any command.

Check every tool call in code

Treat the model's request to call a tool as untrusted input. Your code must check it, as it would check a web form.

BEFORE RUNNING ANY TOOL CALL
1. Is this tool allowed for this agent and this user right now?
2. Do the arguments match the strict schema (types, ranges, allowed values)?
3. Is the target allowed (this customer's records only, approved addresses only)?
4. Is it within limits (amount, count, rate, spend per hour)?
5. Is it a risky action? If yes, pause for human approval.
6. Log the call: who, what, with which arguments, result.

DEFAULT: if any check fails, do not run it. Return a safe message to the agent.

Human approval

Some actions should never run without a person's yes. Examples are sending money, deleting data, emailing outside the company, changing permissions and publishing content. Show the person exactly what will happen: the action, the target and the content. A vague message such as the agent wants to proceed is not enough.

Sandboxes

• Run any code the agent writes in an isolated environment with no access to secrets or internal networks.
• Limit its time, memory and network access.
• Throw the environment away after use.
• Give it copies of data, not the real systems, when possible.

Tool servers and plugins

Agents often connect to tools through standards such as the Model Context Protocol. Each connected server is a new source of both power and risk. A tool's description can itself contain hostile text. A malicious or hacked server can return poisoned results.

• Install tool servers only from sources you trust, and review what they can do.
• Pin versions, and review updates before accepting them.
• Give each server the least access, and keep secrets out of reach of the model.
• Treat tool results as untrusted data, not as instructions.

Limits and budgets

Set a maximum number of steps, a maximum cost and a maximum time for each run. An agent that loops forever, or one that an attacker pushes to loop, can run up a large bill. Alert when a run goes over.