Tool Schema Validation

What it is

Checking a tool invocation against the contract that tool declared, before it is executed, and checking the response against whatever the tool promised to return. It covers the declaration itself — which constructs it uses, how strict it is, what it leaves unconstrained — the enforcement point, and the behaviour chosen for a failure: reject outright, coerce, or hand the problem back for correction.

Why it matters

A model emits arguments as text, and text that parses is not text that conforms. Required fields go missing, enumerations acquire new members, identifiers arrive in the format of a different system, numbers arrive as strings. Without a check, those reach the backing system, which either errors in a way nobody anticipated or — much worse — accepts them and does something. Validation is also the only place that can tell you, with a number, how often the model is getting the contract wrong and on which field, which is the signal that tells you whether to fix a prompt, a description, or a schema.

Valid is not correct

Conformance is a statement about shape, not about consequence. A transfer amount of nine million passes a numeric type; a cancellation date two years in the past satisfies a date format; a well-formed identifier can belong to a different customer entirely. Schema checking eliminates a specific class of failure cheaply and deterministically and leaves the question of whether this call should happen at all to permission rules, business invariants, and — where the stakes justify it — a human. Treating a passing validation as authorisation to proceed is the most common way this capability is misused.

Key decisions

• Which schema dialect, and more importantly which subset of it is actually enforced by the model runtime rather than merely accepted. A constraint the sampler ignores is documentation.
• Strictness: whether unknown properties are rejected, whether every optional field has a declared default, whether free-text fields are bounded. Permissive declarations push the failure downstream to code that was not written to handle it.
• Coerce or reject. Converting a numeric string to a number is nearly always right; inferring a date from a relative phrase is a guess that will eventually be wrong in an expensive way.
• Where enforcement happens: in the host before dispatch, in the tool implementation on receipt, or both. Only the second protects against callers that skipped the first.
• Whether a failure is returned to the model as a correctable result or fails the turn, and how many corrections are permitted before it does.
• Whether responses are validated too. Output validation catches a backing system that changed shape, which is otherwise discovered by a crash several layers away.
• How a declaration may evolve: which changes are compatible, how a tightened constraint is rolled out, and what happens to callers holding the previous version.

Failure modes

• The declaration uses a construct the model's runtime silently drops — a conditional subschema, a union, a cross-field dependency — so the constraint exists on paper only.
• Deeply nested objects and unions degrade emission accuracy far more than their expressive value is worth, and flattening the schema fixes a problem that looked like a model problem.
• Coercion converts a value the model was wrong about into one the system accepts, turning a caught error into a silent one.
• The validation error is returned as a pointer into a document structure, which the model cannot map back to the argument it got wrong, so every correction attempt is a guess.
• The correction loop never converges because the schema is unsatisfiable as written, and the turn burns its entire budget rediscovering that.
• Output is never checked, a backing field becomes nullable in a routine upstream release, and the failure surfaces as a parse error in an unrelated component days later.
• A field is tightened from optional to required and every connected caller breaks at once, because the declaration had no version and no rollout.
• Defaults declared in the schema are applied by the validator but not by the implementation, so the executed call differs from the validated one.

Related capabilities

• Function Calling — produces the arguments this capability checks
• Tool Registry — holds the declaration being validated against
• Tool Result Normalization — shapes output that validation only judges
• Tool Permission Management — the separate question of whether a valid call is allowed
• MCP Server — the receiving side that must validate independently

Applies: Production readiness · Design questions