Evaluation and Quality Engineering
Evaluation Gates — Stack
Evaluation gates › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
What is being chosen is where the condition is expressed and evaluated, and what evidence it leaves behind. The plumbing that runs it is a separate concern; these criteria are about the policy object.
- Condition expressiveness. Whether a rule can reference a baseline, a segment, a severity band and a variance estimate, or only a flat number against a flat metric.
- Threshold provenance. Whether the rule records who set the level, when, on what evidence, and when it is next due for review.
- Override records. Whether an override requires an identified approver and a stated reason, and whether it can be made to expire automatically.
- Missing-result behaviour. Whether the absence of an evaluation result is configurable as a failure, rather than defaulting to a pass.
- Graded consequences. Whether a condition can do something other than block — require a second approver, open a tracked exception, cap the rollout percentage.
- Auditability. Whether every evaluation of every condition is retained with its inputs, which is what makes the control evidence rather than decoration.
- Change control on the rules. Whether editing a threshold goes through the same review as changing the system, since otherwise the gate is advisory with extra steps.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Thresholds expressed in the evaluation suite itself, as assertions that fail the run | The simplest thing that works, with the condition beside the measurement it reads | No override record, no provenance, no graded consequence; editing a threshold is an ordinary code change |
| Policy-as-code engines evaluating a result document (Open Policy Agent and comparable rule engines) | Expressing conditions once and applying them uniformly across many services and release paths | Another language and another system; conditions become remote from the teams whose releases they stop |
| Release and deployment tooling with promotion criteria (progressive delivery controllers, feature-flag platforms with guarded rollouts) | Graded consequences — capping exposure rather than blocking, and automatic rollback on a live signal | Oriented to operational signals such as error rate and latency; quality metrics have to be fed in as custom inputs |
| Gating features of evaluation platforms (LangSmith, Braintrust, Langfuse and comparable) | Keeping the condition next to the run history and the baseline it compares against | Vendor-resident policy; override and audit semantics vary and are worth checking before depending on them |
| Governance and risk platforms holding the conditions as controls | Organisations that must evidence a control to an auditor on a schedule | Produces documentation rather than enforcement unless it is genuinely wired to the release path |
| Change-management workflow tooling (approval records in an issue tracker or service-management system) | Making overrides visible, attributable and followed up with a tracked exception | Entirely out-of-band; nothing stops a release that skips it unless the pipeline checks |
| A results document plus a reviewed checklist | Small teams, and early programmes where the conditions are still being learned | Depends on discipline that will not survive a deadline; no audit trail worth the name |