Guardrails and AI Safety
Hallucination Detection — Stack
Hallucination detection › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
Everything here is a proxy for a property nobody can measure directly, so the criteria are about what each proxy actually detects and what it costs per answer.
- Multiplier on generation cost. Sampling methods cost a multiple of the answer; lookup methods cost a network call; judge methods cost a second generation.
- Signal availability. Whether the serving model exposes token log-probabilities at all, which many hosted endpoints do not, and whether that is stable across providers.
- What it is blind to. Consistency methods are blind to systematic error; lookup methods are blind to anything without a registry; judges are blind to what they also believe.
- Granularity. Whether the output localises a suspect claim or scores the whole response.
- Authority coverage. For referent checking, whether an authoritative source exists for the identifier classes your product emits, and how current it is.
- Determinism. Whether repeated evaluation of the same answer gives the same result, which decides whether it can gate anything.
- Sampling integration. Whether the component can be driven over a slice of recorded traffic rather than inline, which is where most of these belong.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Multi-sample consistency scoring (the SelfCheckGPT approach and its variants) | A reference-free signal that needs no external data and works on any model | Costs several generations per answer, and a confidently repeated falsehood is maximally consistent |
| Token-level uncertainty from log-probabilities | Nearly free when the provider exposes it, and useful for routing rather than deciding | Many endpoints do not expose it; preference tuning degrades its calibration; it measures surprise, not error |
| Verbalised confidence requested from the model | Trivial to implement and occasionally informative on well-studied domains | Systematically overconfident, and the expressed number is itself generated text |
| A second model as judge | Open-ended prose where no registry and no evidence set exist | Correlated errors with the generator, a second generation's cost, and verdicts that vary between runs |
| External verification of referents (package registries, compilers and linters, schema and API catalogues, citation databases, internal identifier stores) | The exactly checkable subset, where precision is effectively perfect | Covers only what has an authority; a stale mirror inverts the result for recently changed entries |
| Trained hallucination scorers (Vectara's evaluation model and comparable published scorers) | A single calibrated number over an answer, cheaply and reproducibly | Most are trained for summary-against-source checking and degrade when no source is supplied |
| Evaluation and monitoring harnesses (DeepEval, promptfoo, Ragas, trace sampling pipelines) | Running any of the above over a sample of production traffic as a quality signal | Measurement tooling used as a gate inherits defaults that were never chosen as policy |
Models
The useful mental model is that every method here is a second opinion, and second opinions are only worth their cost when they are independent. A judge from the same family as the generator has absorbed the same corpus and tends to agree with the same confident errors, so cross-family adjudication is the cheapest independence available and is worth more than a larger model from the same lineage. Sampling-based methods obtain their independence from decoding variance rather than from a different model, which means they measure how stable an answer is and not whether it is correct — a model that has learned a falsehood will state it identically at every temperature, which is exactly the failure that matters and exactly what this family cannot see. Small models can do the adjudication work when the claim is isolated and short, and isolating the claim from the surrounding answer is worth more than model size, because a judge shown the whole fluent response rates its plausibility rather than its truth. Whatever is chosen, insist on two numbers from any evaluation: how many fabrications it finds and how many correct statements it flags, measured on answers your own domain experts have adjudicated. A detector reported without the second number is not characterised at all.