Human Evaluation

What it is

Having people look at what the system produced and record a verdict under a written procedure. It is the only mechanism that can be asked about properties nobody has operationalised — whether this would actually help the person who asked, whether the tone is wrong for a bereavement notice, whether a clinician would act on this — and it is the source of the labels every automated scorer has to be checked against. The engineering problem is not getting opinions; it is making opinions from different people on different days comparable enough to aggregate.

Why it matters

Every other instrument in an evaluation stack is ultimately anchored here. A model judge is trustworthy to the extent it agrees with people; a computable check is useful to the extent somebody confirmed that what it checks is what matters. Human review is also the only mechanism that can discover a failure class nobody has named, because a reviewer encountering something genuinely strange will write it down, and no automated scorer will ever report a category it was not given. The cost is real and the latency is days, which is why it belongs where it is decisive rather than where it is routine.

Key decisions

• Who reviews. Domain experts are scarce, expensive and the only people qualified for anything specialised; a general crowd is fast and cheap and will confidently rate a wrong clinical answer as excellent.
• What the task is. Rating one output on a scale is fast and noisy; choosing between two is far more reliable and yields only a ranking; annotating specific defects is the most informative and the slowest.
• How many reviewers see each item, and how disagreement is resolved — majority, adjudication by a senior reviewer, or treated as a signal that the item is genuinely ambiguous.
• Whether agreement between reviewers is measured and published. Without it, an aggregate is a number with no error bar produced by people who may not have been answering the same question.
• How items are selected for review, since a queue filled with flagged outputs measures the flagging rule, and a uniform sample spends expert time on cases that are obviously fine.
• What reviewers are shown besides the output — the input, the retrieved evidence, the alternative candidate, the model's identity — because each addition changes what is being measured and some of them introduce a bias.
• What reviewers are exposed to, and what support exists for those reviewing harmful content, which is a duty-of-care obligation and not a throughput question.

Failure modes

• Disagreement between reviewers is never measured, and a two-point difference is reported as a finding when reviewers disagree with each other by four.
• The guideline document is edited mid-campaign, and the data before and after is pooled into one average as if the question had not changed.
• Reviewers see the model name beside each output and rate the familiar one higher.
• Rating drifts over a long session as reviewers recalibrate against what they have just seen, so the first hundred items and the last hundred are scored on different implicit scales.
• Throughput is the metric reviewers are managed against, and the scale collapses to whatever value can be selected fastest.
• A five-point scale with no written anchors produces almost nothing but threes and fours, and the aggregate cannot move.
• Review is scheduled at the end of a release cycle, the results arrive after the decision was made, and the programme is cancelled for not being useful.
• The reserved sample used to validate an automated judge is drawn from typical traffic, so it contains almost no hard cases and the reported agreement describes the easy ones.
• Expert reviewers spend their time confirming that obviously correct answers are correct, because nobody triaged the queue before it reached them.

Related capabilities

• LLM-as-a-Judge — the scorer whose validity is defined by agreement with these verdicts
• Golden Datasets — where an adjudicated verdict becomes a durable expectation
• Safety Evaluation — the area most dependent on people and most in need of reviewer protection
• Multi-turn Evaluation — review work that has to be done over whole conversations rather than single outputs
• Online Evaluation — supplies the production sample this review queue is drawn from

Applies: Production readiness · Design questions