Cabo Advisory

← Insights··Updated

Negotiating Hallucination Rates: Accuracy Warranties in AI Service Agreements

Your AI vendor's system can be 99.9% available and still be confidently wrong all day. Traditional SLAs were never built for that — and the contracts are only now catching up.

The problem uptime can't see

A traditional IT service level agreement answers a binary question: is the system up or down? That framing collapses when the service is a probabilistic model. An AI system can meet every availability and latency target while its answers quietly degrade — a phenomenon known as model drift — or while it produces hallucinations: outputs presented as reliable that are materially incorrect, on which a user acts and suffers loss.

For international IT contracts, this creates a genuinely new drafting problem. The thing the customer actually cares about — is the output right? — sits outside the vocabulary of the classic SLA. The market's answer, emerging through 2025 and 2026, is a new layer of contractual machinery: accuracy warranties, hallucination thresholds, and what some practitioners have started calling quality level agreements (QLAs), which extend service commitments from availability into the substantive quality of the output itself.

Why vendors disclaim — and why buyers stopped accepting it

Start with the vendor's position, because it is not irrational. Generative AI is probabilistic by design; no honest provider can promise a zero error rate. Standard AI product terms therefore disclaim accuracy and fitness for purpose almost completely, and vendor-side counsel have good reasons to hold that line: warranting accuracy or guaranteeing outcomes strengthens any later claim against the vendor, and can even complicate the vendor's technology E&O insurance position if the insurer concludes the vendor assumed liability beyond the policy. Well-drafted vendor terms now disclaim the AI failure modes by name — accuracy, completeness, hallucinations, non-infringement, and the effects of model changes — precisely because generic disclaimers have proven fragile.

The buyer's position hardened the moment AI moved into core business processes. As Morgan Lewis has reported from live deal flow, customers embedding AI in consequential workflows are increasingly unwilling to accept a warranty-free posture, and more deals now include performance benchmarks, accuracy thresholds, and obligations to remediate when outputs fall short — even though building meaningful thresholds is acknowledged to be time-intensive. Kemp IT Law framed the same question bluntly in its 2026 outlook: what is an acceptable hallucination rate, and how do you manage it contractually?

Anatomy of an accuracy commitment

A workable AI accuracy framework typically has five moving parts, and each one is a negotiation in itself.

The metric. An accuracy rate is usually defined as the minimum percentage of outputs that are factually correct against a verified reference — a curated knowledge base or an agreed test set. A hallucination threshold is its mirror image: the maximum permissible rate of fabricated or false output. Vague aspirations ("commercially reasonable accuracy") are worth little; specific numbers change behavior.

The baseline. Numbers only mean something relative to a benchmark. Published evaluations suggest that even top-performing models still hallucinate at meaningfully different rates across domains, with rates on specialist legal and medical questions often well above those on general knowledge — which is why thresholds for high-stakes applications are typically negotiated case by case and set well below generic benchmarks. Some internal governance frameworks go further, committing teams or vendors to sub-1% rates on verified test sets, with zero tolerance for certain clinical or safety-critical decisions.

The measurement protocol. This is where most drafts fail. Who runs the evaluation — vendor, customer, or an independent third party? On what test set, refreshed how often, and who owns it? Is measurement continuous (automated evaluation pipelines with alerting when rates creep above threshold) or periodic? Without an agreed protocol, an accuracy warranty is an invitation to a dispute about methodology rather than a remedy.

Model change protection. Accuracy commitments interact dangerously with the vendor's freedom to update models. Insurance case studies in this market already describe the fact pattern: a model update reduces prediction accuracy from 95% to 88%, disrupting a customer's dependent operations and generating a seven-figure breach-of-warranty claim. Customers should tie accuracy commitments to a defined model version or require re-validation and notice before material model changes; vendors should ensure their update rights and their warranties are drafted to be compatible.

The remedies ladder. Because some error rate is inevitable, remedies matter more than the headline number. Emerging market practice builds an escalation: service credits for minor breaches; vendor-funded human review of outputs while accuracy remains below threshold; a formal remediation plan with deadlines; and termination for cause where the system exceeds its hallucination limit persistently — for example, over several consecutive months — or fails an agreed audit. That ladder converts a probabilistic problem into a governed one.

The cross-border dimension

International deployments add two complications. First, regulatory expectations increasingly reference accuracy directly: the EU AI Act's requirements for high-risk systems include accuracy and robustness obligations, and deployers will look to vendors for contractual support in demonstrating compliance — even though the AI Omnibus (Regulation (EU) 2026/1744) has pushed the high-risk deadlines back to December 2027 and August 2028. US state regimes push in the same direction, though the map keeps moving: Colorado's 2024 AI Act, with its duty of reasonable care around algorithmic discrimination, never took effect and was replaced in May 2026 by a narrower, disclosure-based law effective January 1, 2027. Second, an accuracy warranty measured in one language or market may not travel: a model performing at threshold in English may miss it badly in Spanish or Japanese. Multinational agreements should say expressly which languages, regions, and use cases the commitment covers.

Drafting takeaways

For customers: don't ask for a guarantee of truth — ask for a measurable commitment, an agreed test methodology, protection against silent model changes, and a remedies ladder that funds human oversight when the machine underperforms. And keep your own house in order: contract terms cannot prevent hallucinations, so documented human review, fact-checking, and audit trails remain the customer's first line of defense, whatever the warranty says.

For vendors: a bounded, well-measured accuracy commitment on a defined use case is safer than it looks — and increasingly a competitive necessity. The risk lies not in the number but in ambiguity: undefined metrics, unversioned models, and remedies that were never priced. Disclaim what you must, warrant what you can measure, and make sure your insurance program knows the difference.

The uncomfortable truth for both sides is that accuracy warranties are less about allocating blame than about forcing the operational discipline — test sets, monitoring, escalation paths — that production AI needed anyway. The contract is just where that discipline becomes enforceable.

Sources and further reading

This post is general commentary, not legal advice. Specific transactions should be reviewed by qualified counsel in the relevant jurisdictions.