Guardrails and AI Safety
Human Approval Gates — Patterns
Human approval gates › Patterns
Pin the payload, not the intention
Problem. A gate that captures what the agent meant to do, and then reconstructs the action after approval, has approved a description. Any non-determinism between the two — a regenerated argument, a re-planned step, a template filled from fresh state — means the executed action is not the one the person agreed to, and nothing in the system can tell.
Approach. Serialise the exact executable payload at suspension, store it as the approval's subject, and execute that stored object on resume with no regeneration of any kind. Everything that could vary is pinned alongside it: the model and prompt versions, the tool revision, the resolved identifiers. The interface renders from the stored payload rather than from a parallel summary, so what is displayed and what will execute have one source.
Trade-off. A pinned payload grows stale while it waits: an identifier that resolved correctly at suspension may refer to something different, or nothing, by the time it is approved. Rendering directly from the payload also produces interfaces that are honest and hard to read, and the pressure to add a friendly summary alongside it is the pressure that created the problem.
Revalidate at resume, decide at resume
Problem. Everything checked when the gate fired describes a moment that has passed. Between the request and the click, balances move, records are edited or deleted, permissions are revoked, and the person who approved may no longer be entitled to.
Approach. Treat approval as removing one blocker rather than as authorising execution. On resume, re-run the preconditions — the entitlement of the approver, the state the action assumes, the limits it must fit within — and fail the run if any has changed materially, rather than proceeding on the strength of a decision made under different facts. State the staleness window explicitly, and expire approvals older than it instead of honouring them.
Trade-off. A revalidation that fails after a human has already spent attention is a visibly wasted interruption, and frequent failures train approvers to treat the whole mechanism as unreliable. Defining "changed materially" is also a per-action judgement that nobody wants to own, and a conservative definition turns ordinary background activity into constant re-approval.
Budget the interruptions
Problem. Gates are added one at a time, each individually justified, and nobody owns the total. At some firing rate, approvers stop reading and start clicking, at which point the control has inverted: it produces a documented consent trail and no oversight whatsoever, which is worse than having no gate at all.
Approach. Set an explicit budget — interruptions per approver per day — and treat it as a constraint on the system rather than as an observation about it. New gates are funded from the budget, which forces an argument about which existing gate is worth less. Monitor time-to-decision and the proportion of requests approved without the detail being opened, and treat a rising approval rate with falling dwell time as a failing control rather than as an efficient one.
Trade-off. Budgets force gates to be removed from actions that genuinely warrant them, and the removal decision is made on aggregate statistics rather than on the case that will eventually go wrong. Measuring attention is also crude, and the proxies available — dwell time, whether a detail view was opened — are easy to satisfy without reading anything.
The approver is not the requester
Problem. The cheapest routing sends the approval to the person who started the run. That makes the gate fast and makes it meaningless: the user who asked the agent to do something is the user least likely to object to it, and the mechanism records independent review where none occurred.
Approach. Decide per action class whose attention is actually being bought. Self-confirmation is appropriate where the risk is a mistake and the user has the context to spot it. An independent approver is required where the risk is the user's own intent or entitlement, and the system should enforce that the approving identity differs from the requesting one. Record both identities so the distinction is visible in the audit record rather than inferable.
Trade-off. Independent approval needs someone with enough context to judge, and in most organisations that person is also the busiest and least available. Enforced separation also breaks down in small teams and out of hours, where the only person awake is the one who made the request, and the exception path that solves this is where the control usually dies.