Evaluation Gates — References

Material on release controls that hold, on choosing a level that can be defended, and on the governance frameworks that expect such controls to exist.

Controls that degrade

• The reliability-engineering practice of error budgets — a pre-agreed allowance that converts a measurement into an automatic change in release behaviour, rather than into a negotiation — is the closest well-documented analogue to a quality gate, including its treatment of what happens when the budget is exhausted. Described as a practice; the material is set out in site reliability engineering literature rather than in a single paper.
• The continuous-delivery literature on build-breaking checks, and the observation that a check which fires unreliably is disabled in practice long before it is disabled formally, is the standing argument for deriving levels from measured variance. Described as a body of practice rather than attributed to one title.

Why a level has to come from variance

• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (Zheng et al., 2023), arXiv:2306.05685 — cited here because the judge variability it documents sets a floor on the sensitivity of any gate placed on a judge-scored metric.

Governance expectations

• The NIST AI Risk Management Framework — structured around governing, mapping, measuring and managing risk, and the usual vocabulary for describing a release control to people outside engineering. Cited by name.
• Model Cards for Model Reporting — Margaret Mitchell, Simone Wu, Andrew Zaldivar and colleagues, Conference on Fairness, Accountability, and Transparency, 2019. Relevant here for the practice of declaring the conditions under which a system was judged acceptable, which is what a gate's recorded threshold and its provenance amount to. Cited by name.

No verified reference is recorded for graded gate consequences, for override expiry as a control, or for failing closed on a missing evaluation result. These are operational practices carried by delivery tooling conventions rather than by published measurement.