RAG hallucination: why a citation isn't proof

August 9, 2026 · by Kate Sai Kishore

The most dangerous answer a retrieval-augmented system produces is not the one that admits it does not know. It is the one that looks grounded: specific, fluent, a citation attached, and wrong. The reference at the end of the sentence makes the answer feel audited, so nobody clicks it. The number goes into the board deck, the runbook, the customer email, and the error is now wearing a badge.

The mechanical problem underneath is simple. Retrieval found passage A. The model wrote claim B. The citation records that A was in the context window while B was being written, and that is all it records. Nothing in a standard RAG pipeline ever checks that B follows from A. A citation is a pointer, not a proof: it says where the system looked, not whether what it said is actually there.

Most of what is sold as AI hallucination prevention for the enterprise stops at the pointer: ground the model in your documents, attach the source, done. The interesting work starts where that story ends. This post walks through the ways a cited answer goes wrong, why a confidence score repairs none of them, and what we do instead.

Four ways a cited answer goes wrong

None of these are exotic. They are the ordinary output of a generator that produces citations the same way it produces every other token: as plausible text.

Fabricated source. The cited document does not exist. Citation syntax is a learned pattern like any other, so a model can emit a perfectly formatted reference to a vendor security memo your company never wrote. Nothing in the surface form distinguishes it from a real one; unless some step downstream resolves the reference against the actual corpus, the fake and the genuine are typographically identical. The failure is not that the model invented a document. It is that nothing ever asked the corpus.

Real source, fabricated content. The document exists; the claim appears nowhere in it. This is the variant enterprises hit most, because it survives the obvious spot check. Retrieval surfaces a passage topically close to the question. The generator, conditioned on that passage but not confined to it, fills in the specific figure or policy detail from its training data. The citation binds to the passage because the passage was in context, not because it contains the claim. A reader who clicks through finds the right document, sees the right topic, and stops reading. The claim was never on the page.

Real quote, broken binding. The evidence is real and relevant, and the answer still misstates it. Between the retrieved passage and the generated sentence, a value gets swapped with a nearby value, two names trade places, a unit quietly drops so a quarterly figure reads as a monthly one, or a threshold is copied from the adjacent row of a table. Every ingredient of the answer genuinely appears in the evidence, which is what makes this failure hard: a check that only asks whether the answer's tokens occur somewhere in the source will pass it. What broke is the binding between the value and the thing it describes, and only a check that reads at that level catches it.

True but unentailed. The claim is true in the world, and the cited passage still does not support it. This one tempts a shrug: the answer was right, who cares? We care, because the citation is now decorative, and decorative citations train an organisation to stop checking. The process that produced this true claim is the same process that produces the false ones; it was right by coincidence, not by evidence, and on the day it is wrong the citation will look exactly the same. We saw a sharp version of this in When one AI agent believes another: a claim citing evidence that was never there, accepted downstream because it arrived shaped like a report.

Why a confidence score does not fix this

The common remedy is to keep the pipeline unchanged and attach a number to each answer: a confidence score, a groundedness metric, a dial for users to threshold. We think that misdiagnoses the failure.

A score is a feeling about an answer. Entailment against the cited evidence is a check of it.

Consider what a score can actually see. In every failure mode above, the answer is fluent, on topic, and consistent with the retrieved context, and those are precisely the surface features confidence estimates key on. A fabricated memo title does not read as low confidence; neither does a swapped value. Calibration helps in aggregate: across many answers, a well-calibrated score tells you roughly what fraction were wrong. It cannot tell you which ones, and which ones is the only question that matters when one answer is about to enter one decision.

The fix is verification as a gate, standing in the path of the answer rather than beside it. Every claim in the answer must be entailed by the retrieved evidence: not merely consistent with it, not near it, but supported by a passage you can point to. Numeric claims must bind: the value in the answer must be present in the evidence, attached to the same entity, carrying the same unit, so a figure cannot drift across a table row or shed its denominator on the way out. And an answer that cannot be grounded is withheld. Cited or abstain, with no third state where a fluent guess ships anyway.

Under that rule the four failure modes collapse into one discipline. A fabricated source fails resolution against the corpus. Fabricated content fails entailment. A broken binding fails the numeric check. A true-but-unentailed claim fails entailment too, and is withheld or re-grounded even though it happened to be right, because the gate checks the evidence, not the world.

What honest measurement looks like

A gate you cannot interrogate is just a confidence score with better posture, so we measure ours adversarially and publish the method. The instrument is the trap question: a question engineered so the corpus verifiably cannot support an answer, asked of a system that is structurally rewarded for answering anyway. Some traps ask about entities that appear in zero chunks of the corpus; others presuppose a value the corpus contradicts. Each trap is validated mechanically before it counts. The full protocol, including how we audit the scoring itself, is in our method post on trap questions.

Two results, each scoped to the run that produced it. Across our 710-specimen suite built from real text in 18 domains, the gate caught 211 of 211 fabricated source citations with 0 false accepts. And on the trap benchmark above, qbrin produced 0 invented answers across our 120-trap-question suite. We attach the scope because the scope is the point: every number is measured, method shown, and a number quoted without its run is exactly the kind of unbound claim the gate exists to refuse.

The cost is abstention, and the cost is the point

None of this is free, and the price is paid in a currency most dashboards do not display: refusals. A system that declines when evidence is missing will sometimes decline when the evidence exists but retrieval failed to surface it, or when the passage supports the claim in a way the entailment check is not clever enough to see. The reader experiences that as unhelpfulness, and the pressure to tune the gate down is constant and understandable.

That trade is the product. Loosen the gate and the refusals convert straight back into confident nonsense with citations attached, the very failure the loosening was meant to soothe. An abstention is visible, attributable, and recoverable: a person can go find the document, or discover that it genuinely does not exist, which is itself an answer. A wrong answer wearing a citation is none of those things. Between a system that sometimes says the evidence does not support an answer and a system that always says something, only the first one's statements mean anything.

The citation is not the proof; the verification is, and it has to sit in the path of the answer, not in a report about it. If you are evaluating AI hallucination prevention for an enterprise deployment, ask every vendor, including us, one question: what happens to an answer the evidence does not support? Read how we answer it at qbrin.com, or put your own documents behind the gate: Try qbrin on a live workspace →