DL0083 LLM Hallucination and Reduction

A hallucination is fluent, confident model output that is not supported by the provided source or by verifiable fact. The standard split is intrinsic (the output contradicts a document that was placed in the context) versus extrinsic (the output adds unsupported content, such as an invented citation, API argument, or date). Hallucination is structural rather than a bug in any single checkpoint: next-token maximum likelihood rewards plausible continuations, not true ones, parametric memory of rare facts is lossy, and benchmarks scored on plain accuracy give zero credit for saying “I don’t know,” so a guess strictly dominates an abstention during both evaluation and preference tuning. Reduction therefore works on three fronts at once: put the evidence in the context (retrieval grounding with citations), check the produced claims against that evidence (claim-level verification), and change the objective so the model is allowed to abstain when its confidence is below the scoring threshold.

(1) Grounding Beats Recall: retrieving the passage and requiring an inline citation converts a memory problem into a reading-comprehension problem, which is what cuts unsupported claims the most for factual queries.
(2) Verify At Claim Granularity: decompose the answer into atomic claims and run an entailment (NLI) check of each claim against its cited span; a single unsupported claim in an otherwise correct paragraph is exactly what whole-answer scoring misses.
(3) Uncertainty Signals: sampling several answers and measuring disagreement over meanings rather than token strings (semantic entropy, SelfCheckGPT-style consistency checks) flags confabulation without any external corpus.
(4) Calibrated Abstention: a fixed answer-or-abstain threshold derived from the scoring rule trades coverage for precision, and is the only technique that removes hallucinations the retriever never had evidence for.
(5) Decoding And Tuning Help At The Margin: contrastive decoding against earlier layers or against the ungrounded prior, plus fine-tuning that rewards “insufficient evidence” responses, reduce sycophancy and copy-drift but do not manufacture missing knowledge.

Figure 1: Illustrative long-tail QA sweep: grounding and verification push the hallucination rate from 42% to 11% while coverage stays near 95%, and only abstention reaches 4%, at the cost of dropping coverage to 78%.

In production the pieces are wired as one loop rather than as independent tricks: retrieve → generate with citations → extract claims → entail each claim against its cited span, and route anything unsupported to a second retrieval pass or to an explicit refusal. Two details decide whether the loop actually helps. First, the verifier must judge support, not plausibility, so an NLI model or a judge restricted to the cited span is required; asking the same generator “is this correct?” mostly reproduces its original error. Second, a retrieval miss must be visible: if the top-k passages are off-topic and the prompt still demands an answer, the model quietly falls back to parametric memory and attaches a real-looking citation to a fabricated statement, which is the worst failure mode because it survives casual review. Systems that expose this path well (Google’s Vertex AI grounding returns per-claim support scores alongside the answer) make the unsupported fraction a monitorable metric instead of an anecdote.

Figure 2: The verification loop: every atomic claim is entailment-checked against the span it cites, and unsupported claims trigger re-retrieval or abstention instead of being returned with a decorative citation.

Mathematical Formulation:

Where:

• is the expected score of answering under a rule that pays for correct, for wrong, and for abstaining; is the model’s probability that its answer is correct.
• The second line is the abstention threshold: answer only when clears it. Plain accuracy scoring is the case , where the threshold collapses to and guessing is never penalized, which is the incentive that trains hallucination into the model.
• is the prompt and a sampled generation; is a semantic equivalence class of generations grouped by bidirectional entailment, with indexing the distinct meanings observed.
• is the semantic entropy: it is near when all samples paraphrase one meaning and large when the model spreads mass over mutually contradictory meanings, so a high value is a practical confabulation flag even when token-level entropy is low.