AI Is Now an Enterprise Risk Too: How ERM Software Has to Adapt
AI Is Now an Enterprise Risk Too: How ERM Software Has to Adapt
For years, AI showed up in enterprise risk conversations as a solution. It would read documents faster, spot anomalies sooner, and draft reports automatically. All of that is real. But something has quietly flipped. As companies plug language models into customer support, finance workflows, code pipelines, and even their own risk tools, AI has become one of the things that can go wrong. That means enterprise risk management software has to track it like any other risk, and most platforms were not designed with that in mind.
Three new kinds of AI risk
1. Model risk. Any model can drift. The data it sees in production slowly stops looking like the data it was tested on, and its outputs get worse without anyone noticing. Banks have long had formal model risk management for credit and pricing models. Now every company that relies on a classifier or an LLM inherits a version of that problem, whether or not it has a model validation team.
2. Vendor LLM changes. Most organizations do not train their own large models. They call one through an API. That is convenient, but it means the behavior of a critical component can change when the provider updates it. A prompt that reliably extracted contract terms last month might format its output differently next month. Rate limits, pricing, data retention terms, and hosting regions can shift too. This is third-party risk, but with a twist: the "supplier" can change the product without a new contract.
3. Hallucination and over-trust. Language models produce fluent, confident text even when it is not supported by data. In a marketing email that is embarrassing. In a board risk report, where directors have legal duties to oversee risk, it can be genuinely harmful. The risk is not just the error. It is that people stop checking because the output sounds so sure of itself.
What ERM software needs to do differently
If AI is a risk category, ERM software solutions need some new capabilities, or at least need to use old ones more deliberately.
- An AI inventory inside the register. Each model or AI service the business depends on should have an entry: what it does, who owns it, which processes rely on it, and what happens if it fails or degrades.
- Indicators for model health. Key risk indicators should not be limited to liquidity or open vulnerabilities. Acceptance rates for AI suggestions, error rates on spot checks, latency, and cost per task are all useful signals that a model is behaving differently.
- Vendor change tracking. When an LLM provider announces a model update or a change to data terms, that event should flow into the platform and trigger a review, just like a supplier's financial warning would.
- Explainability and lineage. Any AI-generated score change, mapping, or narrative should show its inputs. If an auditor cannot trace why a risk moved, the output is not evidence.
- Human sign-off by design. AI should sense and draft. Named people should decide. Every accept or reject should be logged.
- Fallback plans. If a model is switched off tomorrow, what manual process takes over? That plan should live next to the risk entry, not in someone's head.
The interesting part is that none of this requires exotic technology. It requires treating AI with the same discipline already applied to financial controls and critical suppliers. The tricky part is plumbing: connecting the platform to where models actually run, where their logs live, and where vendor notices arrive. That is usually an LLM integration and data engineering job more than a model-building one.
Why this matters for the tech-curious
If you build or tinker with AI tools, this shift is worth paying attention to. The question your work will increasingly face inside companies is not "is it clever?" but "can we govern it?" Tools that expose their sources, log their decisions, and degrade gracefully will be adopted. Tools that act like black boxes will be blocked by risk and audit teams, no matter how impressive the demo.
For a broader look at how modern platforms are architected, including cost ranges and a build vs buy vs extend framework, see TechCirkle's full guide to ERM software solutions.
Frequently Asked Questions
What is model risk in simple terms?
Model risk is the chance that a model gives wrong or degraded results, or is used for something it was not designed for, leading to bad decisions. Drift, poor data, and misuse are common causes.
Why are vendor LLM updates a risk?
When a provider updates a hosted model, its outputs, formatting, pricing, or data terms can change without your code changing. Processes that depended on the old behavior can quietly break.
How can a company track AI risks in ERM software?
By listing each model or AI service in the risk register with an owner, linking it to the processes that depend on it, monitoring health indicators, and recording vendor changes and fallback plans.
Can AI still be useful inside risk management tools?
Yes. AI is helpful for reading documents, flagging unusual indicators, and drafting reports, as long as its outputs are explainable and reviewed by accountable people.
Who should own AI risk inside a company?
Usually a named business owner for each model, with oversight from the risk function and periodic review by internal audit, similar to how other critical systems are governed.
Lo de que un prompt que extraía términos de contratos un mes puede devolver otro formato al siguiente por un update del proveedor es exactamente el tipo de cosa que nadie monitorea hasta que explota. ¿Tenés algún ejemplo real de empresa que ya esté metiendo inventario de modelos en el registro de riesgos o todavía es teoría?