qbrinHallucination cascade: how one bad claim becomes ten agents’ consensus
A claim invented by one agent does not stay one agent’s mistake. It becomes the next agent’s input, then part of a summary, then a vote. Three hops later it reads as independent agreement, and every agent except the first behaved correctly. This post walks through that mechanism, why the standard defenses underdeliver, and the boundary check that stops it.
A hallucination in a team is an input, not an answer
In a single-agent system, a hallucination is one wrong answer. The model asserts something false, a person reads it, and either catches it or does not. The failure has a fixed size: one output, one reader, one chance to check.
In a multi-agent system, the same hallucination is something else: an input to other agents. When agent 1 reports a finding, agent 2 does not re-derive the claim. It conditions on it. Agent 2’s own output now assumes the claim is true, usually without restating where the claim came from. Agent 3 conditions on agent 2, and so on down the graph.
By the time the claim has passed through three agents, it has been paraphrased into new wording, aggregated into summaries alongside genuine findings, and stripped of its provenance. Anyone sampling the system at that point, human or agent, sees the same fact stated in several places in several phrasings. That looks exactly like independent agreement. It is one claim, echoed. This is the hallucination cascade, and its defining property is uncomfortable: the claim becomes more credible as it travels, precisely because traveling strips the details that made it checkable.
One claim, ten agents: a worked example
Here is a constructed example. It is hypothetical by design, a support-triage swarm we made up to show the shape of the failure, so nothing in it is a measurement.
The swarm has ten agents: an intake agent that reads new tickets, a retrieval agent that pulls related history, a summarizer that writes the morning digest, six triage agents that route and prioritize, and an orchestrator that reports upward. Asked about a spike of crash reports, the retrieval agent states that the crash matches a known issue resolved in an earlier release, and attributes this to a changelog entry. No such entry exists. The retrieval step came back thin, and the model filled the gap with something plausible.
Watch what each hop does to the claim. The summarizer folds it into the digest as “a known, fixed issue”: the attribution to a changelog drops out, because summaries compress. The six triage agents, working from the digest, mark incoming crash tickets as duplicates of a resolved bug: for them the claim is background fact, not something to verify. The orchestrator’s end-of-day report states, in its own voice, that the spike was traced to a fixed regression. Ten agents touched the thread. One invented the claim. Nine reasoned correctly from their inputs, which is exactly why nothing flagged it: after the first hop there is no lie left to detect, only sound inference from a poisoned premise.
Why cascades amplify instead of damping out
Intuition says errors should wash out in a crowd: more readers, more chances to catch the mistake. In agent pipelines the opposite happens, for four mechanical reasons.
Provenance stripping. Summaries drop the “according to” chain. A statement that entered the system as “the retrieval agent says the issue was fixed, citing a changelog” leaves the summarizer as “the issue was fixed.” Once the qualifier is gone, downstream agents cannot re-check what they can no longer locate, and the claim’s uncertainty is no longer written down anywhere.
False corroboration. N restatements of one claim read as N observations. Plain text carries no marker that distinguishes three agents who each observed something from three agents who repeated one agent. Any process that counts mentions, a digest, a retrieval index, a reviewer skimming a thread, sees breadth where there is only an echo.
Authority laundering. A claim repeated by an orchestrator inherits the orchestrator’s trust. Readers, human and machine, weight statements by who makes them. When a coordinator restates a worker’s unverified claim in its own voice, the claim is reissued at the coordinator’s credibility, with the original hedges gone.
Consensus pressure. Voting and aggregation schemes treat correlated errors as independent votes. The arithmetic behind majority voting assumes voters fail independently. Agents that share a base model, a prompt style, and, above all, an upstream input do not fail independently. When six triage agents read the same poisoned digest, their unanimous vote carries the evidential weight of one vote.
Why the obvious fixes underdeliver
Majority voting is the first fix most teams reach for, and it fails for the reason above: it buys nothing when errors are correlated, and a cascade is a machine for correlating errors. By the time the vote is taken, the shared input has already fixed the outcome.
Critic agents, reviewer models that inspect other agents’ outputs, help less than they appear to. A critic reading prose judges what prose exposes: fluency, internal consistency, plausibility. A cascade-borne claim scores well on all three, because it was produced by the same process that produces good text. The critic has no access to the one question that matters, whether the evidence behind the sentence exists, and it usually reads the same contaminated context as everyone else.
Confidence ratings compound the problem in a quieter way: confidence propagates as content. Ask agents to attach a confidence estimate and the estimate travels with the claim like any other text. Downstream agents do not discount an inherited rating; they repeat it, and aggregation then averages restatements of a single guess as though they were independent estimates.
We have measured one version of this trade, in simulation, in our drone-swarm study, When one AI agent believes another. There we ran a rule engine over 2,591 inter-drone messages, with three protections on every claim: minimum sweep coverage, a recency requirement, and repeated claims dying if the original was rejected. As that post reports, it still let 58.7% of false claims through and rejected 39.8% of the true ones. Both figures are scoped to that simulation and that rule set, but the direction is the lesson: checks that inspect the surface of a claim pay for a weak catch rate with real losses of true reports.
The structural fix: claims move with evidence, or they do not move
The property a multi-agent system needs is blunt: a claim does not pass between agents unless it carries its evidence. Not a score, not a reputation weight. The evidence itself, or a pointer that resolves to it.
Verification then happens at the boundary, at the moment a claim leaves one agent and enters another, and it asks three mechanical questions. Does the cited evidence exist: is there a document, a log line, a record at the end of the pointer? Does the evidence entail the claim: does the source say what the claim says, not something adjacent to it? Do the numbers bind: does every figure in the claim appear in the evidence with the same value, the same unit, and the same referent?
A claim that passes moves, and moves with its citations attached, so the next boundary can ask the same questions again. A claim that fails is not downgraded or flagged for later: it does not move, and the receiving agent never conditions on it. This treats an agent’s claim to another agent exactly like a model’s answer to a person: cited or abstain. We described that contract, and how we audit it, in 0 invented answers across 120 trap questions. An inter-agent message deserves the same bar, because its reader is less skeptical than a human, not more.
Run the four amplifiers against this boundary and each loses its grip. Provenance cannot be stripped, because provenance is the payload. False corroboration collapses, because restatements share one citation and are countable as one source. Authority laundering fails, because the orchestrator’s restatement either carries the same evidence or does not move. And consensus regains meaning: every claim that reaches a vote traces to evidence, and votes that trace to the same evidence are visibly one observation, not several.
What it costs, and what it buys
We will not pretend the boundary is free. Every hop pays latency for the check. And a verifier that demands evidence rejects some claims that are true but unevidenced: an agent that is right for reasons it cannot cite is stopped alongside the fabricator. The drone study above puts measured numbers on that trade in one setting, and the cost is real.
What the price buys is the property the cascade destroyed: agreement that means something. With verification at every boundary, consensus is corroboration, distinct evidence converging on one conclusion, rather than repetition amplified through summaries and votes. A bad claim can still be generated. It can no longer become ten agents’ consensus, because it cannot cross the first boundary without evidence it does not have.
This is how we build governed agents: every claim an agent makes, to a person or to another agent, passes the same gate. Cited, or it does not move.
Comments
Sign in with GitHub to reply. Threads live in a public repository, so anyone can read them without an account.