Tool Guardrails — Patterns

Judge the effect, not the arguments

Problem. The arguments describe a request; the consequence depends on the target's current contents. A filter expression, a query, a glob, or a selector that looks modest can match everything, and no inspection of the literal values reveals that.

Approach. Require integrated systems to expose a preview — a count of matched records, a diff, a plan, a quoted price — and make that preview the object the rule evaluates. The invocation then carries both what was asked and what it would do, and limits are written against the second. Where a system cannot preview, say so explicitly in its registration and treat the absence as a reason for a tighter static bound rather than as a reason to skip the check.

Trade-off. Two round trips per call, with a window between them in which the target can change, so the preview is evidence rather than a guarantee. Some previews are also as expensive as the operation, and a few have side effects of their own, which makes this pattern unavailable for exactly the heaviest calls.

Budgets scoped to the run

Problem. Per-call limits are blind to repetition. An agent that may send one message may send nine hundred, and every individual call satisfies every rule. The damage is in the aggregate, and the aggregate exists nowhere in a stateless check.

Approach. Hold counters keyed to the run, the session, and the tenant in one shared store, increment them at dispatch rather than at completion, and evaluate limits against the projected total. Express the budgets in units the business recognises — currency, external recipients, rows written — rather than in call counts, so that a limit means the same thing after a tool is refactored. Exhaustion is a defined outcome the agent is told about, not an error.

Trade-off. A shared counter is a hot dependency on the dispatch path, and a strongly consistent one is a latency cost on every call. Increment-before-execute also overcounts whenever calls fail, so budgets drift conservative over a long run unless failures release their reservation, which reintroduces the race the design was avoiding.

One dispatch point, no exceptions

Problem. Checks accumulate around the paths people remember. The paths they forget — a retry worker, a scheduled job, an evaluation harness replaying recorded runs, a debugging endpoint — reach the same systems with the same credentials and none of the rules.

Approach. Make invocation possible through exactly one function, give it the check, and remove every other way to reach an integration from application code. Enforce it structurally where the language allows: unexported clients, a linter rule, a network policy that only the dispatcher can satisfy. Treat the appearance of a second path as a defect of the same severity as a missing rule, because it is the same defect.

Trade-off. A single chokepoint is a single point of failure and a bottleneck for anyone who needs behaviour it does not support, which generates steady pressure for exemptions. It also centralises latency and failure: everything that was independent now shares a dependency.

A recall window instead of a refusal

Problem. Many consequential actions are not wrong, merely unverified, and refusing them stops work that would almost always have been fine. Allowing them makes a mistake permanent. Both answers are poor for the same class of call.

Approach. Let the call proceed into a reversible state rather than a final one: an outbound message queued with a delay, a deletion marked rather than executed, a change staged for commitment. The guardrail's verdict chooses the window length, which may be zero for routine calls and minutes or hours for unusual ones, and the window is what gives detection and human review somewhere to land.

Trade-off. The product now has two states for everything and must reconcile them, which leaks into every interface that reports what has happened. A window that nothing watches delivers no safety at all and simply makes the system slower, and some effects — a third-party API call, an external notification — have no reversible form to stage.