Hallucination Detection — Patterns

Verify the referents, leave the prose alone

Problem. General-purpose detection over free text is expensive, unreliable, and produces a score nobody can act on. Meanwhile a substantial share of the fabrications that actually cause damage are not subtle at all: a library that does not exist, a method that was never on that class, a case citation with no such case, an internal identifier that matches nothing.

Approach. Extract the typed referents from the output — package names, symbols, file paths, identifiers, citations, URLs — and resolve each one against the authority that owns it: the registry, the compiler, the schema, the catalogue, the database. An unresolvable referent is a fact rather than a score, and it can be removed or corrected without any judgement about the surrounding text. Build the extractor per referent class, because the extraction is the only part that needs care.

Trade-off. Coverage is limited to classes with an authority, which means the pattern says nothing about the reasoning, the recommendations, or any claim about the world. Authorities are also not always current: a private registry mirror, a cached schema, or a citation database with a publication lag will report correct new referents as fabrications, and that error is confidently wrong in the opposite direction.

Sampled agreement as a monitor, never as a gate

Problem. Generating several answers and comparing them is the best-known reference-free signal and costs several times the price of the answer. Applied inline it is unaffordable; abandoned entirely it leaves no quality signal at all for open-ended traffic.

Approach. Run it asynchronously over a random sample of recorded requests, with the sample drawn across the whole traffic distribution rather than from whatever is convenient. Track the disagreement rate per request class over time as a quality indicator, alert on movement rather than on absolute level, and use it to decide where to spend the expensive inline checks. The sample is a thermometer; it is never the thing deciding whether a particular answer ships.

Trade-off. A monitor tells you the rate and never protects the individual user whose answer was wrong, which is an uncomfortable thing to explain. Drawing a representative sample is also harder than it looks: the requests most likely to produce fabrication are long-tail ones, and a uniform sample barely includes them.

Isolate the claim before adjudicating it

Problem. A judge shown a complete, fluent, well-structured answer is being asked to resist its presentation. Coherence, confidence, and length all push a judgement towards acceptance, and the judge's verdict ends up tracking how good the answer sounds rather than whether the statement is right.

Approach. Extract each claim, strip the surrounding narrative, and present it alone, without the question, without the rest of the answer, and without any indication that a model produced it. Ask for a verdict on that statement as a standalone proposition. Batch the claims so the cost stays proportionate, and keep the mapping back to the sentence so a finding can be acted on.

Trade-off. A stripped claim loses the context that made it correct — a statement true of one version, one jurisdiction, or one customer becomes false when the qualifier is removed — so this pattern generates a particular kind of false positive that is tedious to triage. It also cannot evaluate anything whose correctness is a property of the argument rather than of any single sentence.

Spend verification where being wrong is expensive

Problem. Checking everything is unaffordable and checking nothing is negligent, and the usual compromise — check a fixed fraction at random — spends the same effort on a throwaway draft as on the figure someone is about to put in a filing.

Approach. Route verification by the consequence of the claim rather than by the suspicion attached to it. Claims that will be acted on automatically, quoted to a customer, written into a record, or used in a calculation get checked; claims in exploratory, clearly-labelled, easily discarded output do not. The routing input is the request's purpose and destination, which the product knows, rather than any property of the text, which it does not.

Trade-off. It concedes that low-consequence surfaces will carry unverified fabrications, which is a defensible engineering position and a difficult one to state publicly. Consequence is also routinely misjudged: output generated as a draft is copied into a document that matters, and nothing in the system knows that happened.