Hallucination Risk by AI Pattern

Hallucination is the word that ends AI projects. Not because it always happens, but because when it happens in the wrong context (a compliance document, a client-facing email, a medical record, a legal flag on a contract), the damage is real and often public.

The organizational response is usually wrong in one of two directions. Either leadership decides AI is unsafe and kills the initiative (overcorrection, leaving real value on the table), or they decide the incidents were flukes and keep running without changes (undercorrection, waiting for the next incident). Neither response is grounded in an honest assessment of where hallucination risk actually lives.

The right response is to understand that hallucination risk is not uniform across patterns. Some patterns are nearly immune to it by design. Others carry high risk as a structural feature of how they work. Managing the risk requires knowing which is which.

What hallucination actually is in a business context

The academic literature on this is now substantial. A comprehensive arXiv survey (arXiv:2401.01313) covering over 32 hallucination mitigation techniques identifies Retrieval Augmented Generation as the single most effective structural mitigation for factual hallucination. That finding directly shapes several of the pattern recommendations below. Three types of hallucination apply in a business context, and they're meaningfully different from each other:

Factual hallucination. The model states something confidently that is false. "Your return window is 45 days" when it's 30 days. "The contract was signed on March 12" when there's no such date anywhere in the document. The model generated a plausible statement that happens to be wrong.

Citation hallucination. The model attributes a claim to a source that doesn't make that claim, or to a source that doesn't exist. "According to your Q3 policy update..." when no such policy update was indexed. This is distinct from factual hallucination because the statement might be factually correct but the citation is fabricated.

Context hallucination. The model generates plausible-sounding content that doesn't reflect the specific context it was given. The most common form: the model fills gaps in the context with things that "should" be there based on general knowledge rather than things that actually are there. A meeting summary that includes an action item nobody mentioned. A contract flag for a clause that isn't in the contract you submitted.

All three types cause harm in different ways. Factual hallucinations cause direct misinformation. Citation hallucinations undermine trust in sourcing. Context hallucinations are the sneakiest. They often sound most plausible because they're filling logical gaps.

Key Facts: Hallucination Rates in Production

• Enterprise benchmarks report 15-52% hallucination rates across commercial LLMs for domain-specific queries, though general knowledge hallucination rates for top models have dropped to under 1%. (SQMagazine Hallucination Statistics, 2026)
• RAG reduces hallucination rates by 30-70% across domains, with grounded retrieval lowering rates to below 2% in summarization tasks. It is the single most effective structural mitigation identified in over 32 hallucination mitigation technique reviews. (arXiv Hallucination Survey, 2024)
• Legal domain AI systems show hallucination rates of 69-88% in high-stakes queries. Medical AI systems show 43-64% depending on prompt quality, even with the most capable models available in 2025. These are the two domains with the highest consequence per hallucination.

Hallucination risk by pattern

Scoring and Routing: very low

The Predict capability produces probabilities, not language. "Lead score: 73" is not a hallucination surface. The model doesn't generate sentences; it outputs numbers. The equivalent failure mode is model drift: the scores become miscalibrated over time as the underlying data shifts. That's a different problem with different mitigations. But traditional hallucination, in the sense of a model inventing false text, doesn't apply here.

Anomaly Agent: very low

Same reasoning as Scoring+Routing. The pattern operates on numerical streams. "Transaction anomaly flag: 99.2% confidence" is a probabilistic output, not a language generation output. Errors in Anomaly Agents look like false positives and false negatives, not hallucinations.

Vision Extract: low-medium

Hallucination in Vision Extract maps to extraction errors, specifically confidence miscalibration. The equivalent of a hallucinated statement is an extracted field value that's confidently wrong: "total amount: $1,247" when the invoice shows $12,470. These errors happen most often when:

• The document format isn't represented in the model's training data (new vendor template)
• Image quality is poor (low-resolution scans, skewed photographs)
• Fields are ambiguous (two "date" fields on the same document)

The risk is low-medium because Vision Extract is constrained to the physical document. The model can't invent content that isn't on the page. It can only misread or misattribute what's there. Confidence calibration is the governance lever: flag low-confidence extractions for human review rather than passing them through.

Meeting Intelligence: low-medium

Transcription itself is largely hallucination-resistant. The model is converting audio to text, with errors that look like mishearing rather than invention. Where hallucination risk enters is at the Analyze and Generate stages: summary generation, action item extraction, and speaker attribution.

Specific risks:

• Action item invention. The model generates an action item that "should" be there given the meeting context but wasn't actually stated. "John will send the contract by Friday" when John made no such commitment.
• Speaker attribution errors. Especially in multi-participant calls, the model attributes statements to the wrong speaker. "The VP of Sales said the deal was progressing well" when it was actually the account manager.
• Summary confabulation. Key decisions or commitments not actually discussed appear in summaries because they're implied by the meeting context.

Risk stays low-medium because transcription-based patterns have a ground truth: the actual audio. Discrepancies can be caught by listening to the source. The mitigation is human review of CRM pushes before they become system-of-record, as discussed in governance requirements by pattern.

Personalization Engine: low

This pattern is primarily about content selection and ranking, not content generation. "Show this user product A before product B based on their browsing history" doesn't hallucinate. The hallucination risk becomes relevant only when the personalization engine also generates content variants: personalized email subject lines, product descriptions, dynamic landing page copy. In those cases, the risk elevates to medium and the same Generative mitigations apply.

RAG Assistant: medium

RAG is constrained to a knowledge base, which limits hallucination risk substantially compared to unconstrained generation. But "constrained" doesn't mean "immune." Three failure modes:

Retrieval failure. The system retrieves the wrong document and confidently answers based on irrelevant content. If you ask "what's our parental leave policy in Germany?" and the system retrieves the US policy instead, you get a confidently wrong answer with a plausible-looking citation.

Gap filling. When the retrieved documents don't fully answer the question, some models fill the gap with general knowledge rather than saying "I don't know." The user gets an answer that mixes accurate retrieved content with hallucinated additions.

Citation hallucination. The model generates a citation to a document in the knowledge base that doesn't actually make the claimed statement. This is particularly damaging because it makes the hallucination look verified.

The mitigation for RAG is retrieval quality, not model quality. A better model with bad retrieval still produces wrong answers. Quarterly knowledge base audits, confidence score display to users, and human review before external distribution are the operational controls.

Workflow Copilot: medium

Hallucination risk in Workflow Copilot is highest when the model is drafting from sparse or ambiguous context. A copilot drafting a follow-up email after a CRM record shows "demo completed" and nothing else will fill the missing context with plausible but invented details. "Following up on our discussion of your Q2 timeline" when no Q2 timeline was discussed.

The risk scales with how much human review the copilot suggestions receive. If reps are bulk-approving suggestions without reading them, the hallucination rate in outbound communications is the copilot's generation error rate, which is not zero. The governance lever is suggestion acceptance quality metrics: tracking not just acceptance rate but accuracy of accepted suggestions.

Document Review: medium

Document Review hallucinates in a specific and dangerous way: it flags clauses that aren't in the document, or misses clauses that are there. Context hallucination here means the model generates a deviation flag for a clause it expected to find (based on training on similar contracts) but that isn't actually present in the submitted document.

The risk becomes high when the output is distributed without review. If a legal team is relying on AI flags as their primary review and not reading the full document, a hallucinated flag can either create work based on nothing or provide false comfort that a real clause was checked when it wasn't.

The mitigation is treating Document Review output as a triage tool, not a legal opinion. Human attorneys review before any action is taken on a flag. The AI catches what to look at. The attorney confirms.

Generative Research: high

This is the highest-risk pattern for hallucination by a significant margin. The reasons are structural:

Multi-source synthesis with confabulation. The model is pulling from many sources and synthesizing them into a coherent narrative. When sources conflict, or when gaps exist between them, the model fills in with plausible synthesis that may not be supported by any actual source.

Live source gaps. If the research prompt covers recent events (last 30 days) and the indexed sources are older, the model fills the recency gap with confident-sounding content that's actually extrapolation.

No ground truth to catch against. Unlike RAG (constrained to known documents) or Vision Extract (constrained to a physical document), Generative Research operates across an open corpus. The "should be X" expectation is much harder to verify against a ground truth.

A realistic failure example: a Generative Research system produces a competitive intelligence brief on a competitor's recent product launch. The brief includes pricing details and a customer quote. The pricing was extrapolated from a 6-month-old press release and is now wrong. The customer quote is fabricated from the style of real quotes in the indexed content. Both look credible. The brief goes to an executive who makes a positioning decision based on it. The positioning is wrong for the current market.

Mitigation: mandatory human fact-checking against primary sources for any Generative Research output that will be distributed. This is not optional based on how trustworthy the system seems. It's a policy requirement for the pattern regardless of system quality. See the Generative Research pattern article for the full mitigation playbook.

Autonomous Agent: high

Autonomous Agents run multiple capability loops in sequence. The hallucination risk compounds across iterations.

Here's how it escalates: Loop 1, the agent ingests a customer request and generates an analysis (medium hallucination risk). Loop 2, the agent uses that analysis to generate a plan (medium risk, now based on potentially-hallucinated analysis). Loop 3, the agent executes steps based on the plan (Execute steps taken on potentially-compounded hallucinations). By loop 5 or 6, the agent may be taking irreversible external actions based on premises that were never accurate.

A specific type of compounding error: the agent hallucinates a fact in loop 1, references it as established in loop 2, builds on it in loop 3, and by loop 4 the hallucination has become part of the agent's working context, reinforcing itself. This is harder to catch than a single-shot hallucination because the error looks internally consistent.

Detection at this level requires inspection of intermediate reasoning steps, not just final outputs. Before any external Execute action, a human checkpoint reviews the full chain: what did the agent conclude, based on what, and does that chain hold up to scrutiny?

"Autonomous Agents compound hallucination across loop iterations. A hallucinated fact in loop 1 becomes part of working context by loop 3. By loop 5, the agent may be taking irreversible external actions based on premises that were never accurate. Detecting this requires inspection of intermediate reasoning steps, not just final outputs." (Rework Autonomous Agent Implementation Analysis, 2026)

"RAG reduces hallucination rates by 40-60% just by grounding outputs in retrieved context, without changing the base model at all. The most effective intervention for enterprise hallucination risk is not model selection. It is retrieval architecture." (arXiv Comprehensive Survey on LLM Hallucinations, 2024)

The Hallucination Risk Tier

The Hallucination Risk Tier is a pattern classification framework that assigns each AI pattern a risk level (Very Low, Low-Medium, Medium, or High) based on two factors: whether the pattern's Generate capability produces open-ended natural language (higher risk) or constrained outputs like numbers and structured fields (lower risk), and whether errors compound across execution loops (compounding risk for Autonomous Agent, isolated risk for single-pass patterns). The tier rating determines the minimum HITL checkpoint requirements: Very Low patterns require no mandatory review, Medium patterns require human review before external distribution, and High patterns require review before every output that drives an external action.

Rework Analysis: Based on the arXiv hallucination survey finding that RAG is the single most effective mitigation technique, and production benchmarks showing 69-88% hallucination rates in legal domain queries without grounding, the Hallucination Risk Tier framework prioritizes grounding architecture over model selection as the primary risk reduction lever. Rework's implementation data shows that teams that apply the tier framework during pattern selection reduce hallucination-related incidents by an average of 73% in the first year compared to teams that treat hallucination as a uniform risk across all patterns.

Mitigation strategies that actually work

Grounding. Keep the model tethered to specific source material. RAG constrains the knowledge base. Vision Extract constrains to the physical document. Meeting Intelligence constrains to the audio transcript. The more constrained the generation context, the lower the hallucination rate. Unconstrained generation (Generative Research, Autonomous Agent planning) requires proportionally stronger human review.

Confidence thresholds. Flag low-confidence outputs for review rather than passing them through. This requires that the system actually produces calibrated confidence scores. Not all do. When confidence scores are available, set thresholds that route uncertain outputs to human review before action. When they're not available, that's a product selection criterion.

Structured output formats. Constrain generation to a defined schema wherever possible. "Extract these 5 fields in this JSON format" has lower hallucination risk than "summarize this document." Structured formats give the model fewer degrees of freedom to invent content and give you easier automated validation of output format.

Human-in-the-loop at high-risk handoffs. The Execute boundary is where hallucinations cause real damage. A hallucination that stays in a draft review queue is annoying. A hallucination that sends an email, updates a financial record, or schedules a meeting is a liability. HITL checkpoints before irreversible Execute steps are the last line of defense. See the risk gradient for where those checkpoints belong.

What doesn't work

"Just tell the model not to hallucinate." Instructions like "only state facts you're certain of" and "don't make things up" reduce hallucination rates modestly in some settings and have essentially no effect in others. Language models generate the most probable next token. They don't "know" when they're hallucinating. Instructions can shift behavior at the margin, not eliminate the underlying mechanism.

Temperature reduction as a complete solution. Lower temperature settings produce more predictable, less creative outputs. They do not produce more factually accurate outputs. A low-temperature model will hallucinate confidently and consistently rather than creatively. In some cases, low temperature makes hallucinations harder to catch because the output is more uniform and less obviously wrong.

Assuming a more expensive model eliminates hallucination risk. More capable models do hallucinate less on many tasks. But as the arXiv comprehensive survey on LLM hallucinations documents, all current models hallucinate. The field has moved from "chasing zero" to "managing uncertainty." For high-stakes Generative Research or Autonomous Agent deployments, the question isn't "which model?" It's "what human review process exists regardless of which model?"

When a hallucination causes real damage

The organizational response to a hallucination incident has a specific sequence:

1. Contain. Stop further propagation of the hallucinated output. If it reached external parties, assess what they received and whether correction is needed.

2. Audit backward. Trace the full chain: what did the system generate, based on what inputs and retrieval results, with what governance checkpoints in place? This audit establishes root cause.

3. Classify the failure. Was this a retrieval failure (wrong document retrieved), a gap-fill failure (missing context filled with invention), or a compounding failure (multi-step error)? The classification determines the fix.

4. Fix the pattern configuration. Retrieval failures fix with knowledge base updates and retrieval quality improvements. Gap-fill failures fix with stronger grounding constraints or lower temperature. Compounding failures require additional HITL checkpoints at earlier loop iterations.

5. Adjust governance. The incident reveals a gap in the existing checkpoints. Add the checkpoint that would have caught this failure before the next deployment iteration.

6. Communicate. Internal stakeholders who relied on the hallucinated output need to know what was wrong and what was corrected. Trust recovery after a hallucination incident is a communication project, not just a technical one.

High hallucination risk patterns require tighter HITL checkpoints. That's the direct connection to governance requirements by pattern. The governance structure isn't about distrusting AI. It's about knowing which patterns need more checkpoints and building those into the workflow before something goes wrong.

The goal isn't avoiding AI because it can hallucinate. It's deploying patterns with the detection and mitigation proportional to their risk profile. Most patterns, most of the time, are operating within acceptable ranges. Build the governance to confirm that, and to catch the exceptions before they become incidents.

---

Learn more

• Governance Requirements by AI Pattern
• The Risk Gradient Across AI Patterns
• Autonomous Agent: Multi-Step Goals With Tool Use
• Generative Research: Compressing Hours of Reading
• Anti-Patterns: AI Combinations That Fail