AI risk management when the AI takes actions
AI risk management is the practice of identifying, measuring and controlling the harms an AI system can cause. For predictive and generative models, that work centres on output quality: accuracy, bias, hallucination, disclosure. For agents, the centre of gravity moves to authority — what the software can reach, what it can change, and whether anything can stop it.
Updated 2026-09-18
The risks that only appear with agents
These have no close analogue in model risk management, because they are properties of software with permissions rather than of a prediction.
- Excessive agency — an agent permitted to do far more than its task requires.
- Orphaned authority — credentials outliving the person, project or purpose that justified them.
- Unsafe delegation — an agent passing work to another agent and widening authority in the process.
- Untrusted content as instruction — indirect prompt injection reaching an agent that holds real permissions.
- Unattributable action — a change in a production system that cannot be traced to an initiating principal.
- Runaway consumption — a loop that spends budget, rate limit or quota without a ceiling.
Why probabilistic controls are not sufficient
Detection is valuable for prioritizing attention. It is a poor basis for permitting an irreversible action, because it is right most of the time rather than all of the time, and the cases it misses are exactly the ones an attacker constructs.
A workable split is to let AI prioritize uncertainty and let deterministic policy constrain consequences. Anomaly scoring can raise an agent for review; it should not be what decides whether a payment executes.
What a control actually requires
For a control to be more than an assertion it needs four properties: a decision point the action must pass through; a policy evaluated at that point using current context; an outcome the system enforces rather than recommends; and a record produced by the decision itself.
The absence of the first is the most common gap. Where there is no chokepoint between the agent and the protected system, there is no control — only monitoring after the fact.
Third-party and supply-chain exposure
Agents are increasingly delivered inside SaaS products rather than built in-house, which shifts part of the risk to vendors. The practical questions are which of a vendor's agents hold credentials in your tenant, what those credentials can reach, whether their actions appear in your audit trail, and whether you can revoke them independently of the vendor relationship.
Common questions
- How is agentic AI risk management different?
- Model risk management assesses outputs — accuracy, bias, disclosure. Agentic risk is about authority and action: what the software can reach, what it can irreversibly change, and whether a control exists that can stop it.
- Can anomaly detection manage AI risk on its own?
- No. Detection is probabilistic and useful for prioritization. Irreversible actions need deterministic policy that does not depend on a model judging intent correctly.
- What about agents inside vendor SaaS products?
- Ask which vendor agents hold credentials in your tenant, what those can reach, whether their actions appear in your audit trail, and whether you can revoke them without ending the vendor relationship.