That autonomy introduces a category of risks that traditional risk frameworks were never built to handle — and most operators only see them after money, personal data, or a regulator is already involved. Here are the risks we encounter most often in our first engagements.
Hallucinations
An agent can generate a factually false statement with full confidence. When that statement feeds an operational decision — an email sent to a customer, a report filed with a regulator, a pricing recommendation — the cost of the error is carried by the business, not the model. Hallucinations are particularly dangerous because they are undetectable without human validation or comparison against a source of truth.
Leaking personal information
Agents with access to customer databases can include personal information in responses, logs, or third-party API calls. The risk is not limited to a mass leak: a single poorly-framed prompt can cause a name, social insurance number, or medical record to travel outside the authorised perimeter. Under Quebec Law 25, this kind of incident triggers a mandatory notification obligation.
Leaking technical secrets
API keys, authentication tokens, database credentials, internal scripts: an agent that has been configured with broad access can expose these elements in its responses or logs. Once disclosed, these secrets are nearly impossible to revoke without operational disruption. Access control must follow the principle of least privilege, agent by agent, action by action.
Procurement errors
When an agent can trigger orders, purchase requests, or contract renewals, a misinterpretation of a request can result in real financial commitments. We have seen cases where an agent placed duplicate orders, selected the wrong supplier, or validated terms that did not match the organisation’s purchasing policy. These errors only surface at delivery or invoicing.
Payment errors
Agents integrated with payment systems can trigger transfers to the wrong recipient, in the wrong currency, or for the wrong amount. Unlike a human error, an agent error can repeat at scale within seconds before an alert is raised. Compensating controls — caps, two-step approvals, beneficiary allowlists — are essential.
Discount and pricing errors
An agent that calculates discounts or dynamic prices can apply incorrect logic at scale. A 90% discount instead of 9%, a price indexed to the wrong currency, a promotion applied to a non-eligible category: each error multiplies by the volume of transactions processed before anyone notices. The remediation cost often includes the goodwill discounts given to customers to preserve the relationship.
Bias
Foundation models inherit the biases present in their training data. When an agent makes decisions that affect people — hiring, credit, service access — those biases translate into systemic discrimination. The legal risk goes beyond reputational risk: Quebec’s Charter of Human Rights and Freedoms and several sector regulations require automated decisions to be explainable and contestable.
Overconfidence and misleading framing
Agents tend to phrase their responses with a level of assurance that does not reflect the model’s actual uncertainty. A user — employee or customer — who receives a confident answer tends to believe it. That dynamic encourages decisions without validation, and hides the areas where the model is actually outside its competence. Calibration controls and explicit caveats are not optional.
And the list keeps growing
We have not covered prompt injection, model drift, single-vendor dependency, tenant contamination, voice-synthesised identity fraud, or cascading errors in agent chains. Every AI deployment introduces its own risk profile — and that profile shifts with every model update, every new integration, every new use case.
Managing agentic risk — what the market today calls agentic AI risk management — is not a checkbox. It is a continuous programme, with a named owner, a live register, tested controls, and an incident response plan. Organisations that treat it as a one-off project will discover their blind spots during an incident — and that moment is generally the worst time to discover a blind spot.
Summary
Autonomous agents introduce risks traditional frameworks never covered — from hallucinations and leaked secrets to procurement and payment errors at scale. The pattern is the same: exposure you cannot see until money, data, or a regulator is already involved. A live register, least-privilege access, and a named owner beat a one-off audit every time.