Policy Enforcement — Patterns

Rules as versioned artifacts with validity windows

Problem. A rule changes and the old behaviour becomes unreconstructable. Six months later someone asks what the system was permitted to say on a particular date, and the only available answer is what the current configuration says, which is not the question.

Approach. Treat the rule set as an immutable, addressable version with an explicit validity window. Every evaluation records the version identifier it used, and the versions are retained for as long as the obligations they encode. A change is a new version with a start date, not an edit; a withdrawal is an end date, not a deletion. Replaying a past decision then means loading the version that was in force and running it against the recorded inputs.

Trade-off. Retaining every version indefinitely accumulates artifacts whose meaning nobody remembers, and replaying an old decision faithfully also requires the inputs, which drags retention of request records along with it. Teams also tend to discover that their current rule set has no coherent first version, and establishing one is an archaeology exercise.

Prefer removing the capability to forbidding its use

Problem. A rule expressed as an instruction depends on the model choosing to follow it, and a rule expressed as an output check depends on recognising the violation in text. Both fail under exactly the conditions that make the rule matter: an unusual request, an adversarial one, or a model update that shifted behaviour.

Approach. For each commitment, ask first whether it can be made structurally impossible rather than forbidden: disconnect the data source rather than instruct the model not to use it, withhold the pricing tool rather than forbid quoting a price, deploy a model variant without the capability for the tenant that must not have it. Reserve instructions and checks for commitments that cannot be expressed structurally, and record which category each rule is in so the weaker ones are visible as a group.

Trade-off. Structural enforcement is coarse and expensive: a capability withheld for one jurisdiction is often withheld in a build or a deployment, which multiplies configurations and slows every release. It also removes the capability from legitimate uses the rule never intended to cover, and reinstating it for a specific case is a far larger change than relaxing a predicate.

Derive the test suite from the prose

Problem. The rule in production and the rule in the policy document diverge immediately and silently. Nobody intends it: the document is edited to clarify an ambiguity, the implementation is adjusted to reduce false positives, and nothing compares them.

Approach. Require every clause of the written policy to arrive with cases — inputs that must be caught and inputs that must not, written by the clause's owner in their own terms and translated into fixtures. Run them in the pipeline, and treat an un-cased clause as unenforced rather than as enforced-by-prose. When a false positive is corrected, the correcting case is added, so the suite accumulates the actual boundary rather than the intended one.

Trade-off. Getting cases from legal and compliance owners is slow and often unwelcome, and the cases they write are frequently ambiguous in ways that only surface when someone tries to make them executable. A growing suite also becomes a constraint on changing the model, because a release that improves everything else can fail on a dozen clause fixtures that encode phrasings rather than meanings.

The evaluation record is the artifact

Problem. Enforcement that only logs violations produces evidence that something was stopped and no evidence that anything was checked. The question asked after an incident is almost always the second one, and a silent pass is indistinguishable from a rule that never ran.

Approach. Emit a record on every request listing the policy version, the rules evaluated, the scope that selected them, and each outcome including the passes. Retain it for the lifetime of the obligation rather than for the lifetime of a debugging log, and give it a schema stable enough that a compliance query can be answered by a person who does not know the codebase. Coverage — the proportion of requests where a given rule was actually evaluated — becomes a monitorable number.

Trade-off. A record per request per rule is a substantial volume with a long retention, and it describes what the system did with content that may itself be sensitive. There is also a real risk of the record becoming the goal: teams optimise for a complete-looking trace while the rules it documents remain thin.