Structured Output Contracts

What it is

The agreement about the shape of what a model returns to the application, and the machinery that makes the agreement hold: a declared schema, a mechanism for constraining or checking generation against it, a defined response to a violation, and a versioning story for when the shape changes. The contract covers the final payload the product consumes — the extracted fields, the classification, the object rendered into a user interface — as opposed to free prose a human reads.

Why it is not tool argument validation

Both involve a model emitting JSON against a schema, and that is where the similarity ends. Tool arguments are checked by the component that is about to execute them, the schema is owned by the tool, and a rejection has somewhere to go: the error is returned into the loop and the model tries again with the failure in front of it, which is ordinary operation rather than an incident. A response contract is the application's own interface to the model's answer, its schema is owned by the consumer, and a violation at the end of a run has no in-loop recipient — it surfaces as a failed request to a caller who was expecting a result. The two also optimise differently: a tool schema is designed to be understood by a model choosing between actions, while a response schema is designed around what the consuming code and the rendering layer need. Argument-side checking is covered by Tool Schema Validation.

Why it matters

A product that parses prose with regular expressions has a parser that fails on the day the model becomes more verbose. A declared contract turns an open-ended generation problem into a constrained one and gives the application a place to stand: code can assume fields exist, interfaces can render without defensive branches, and downstream systems can accept the output without a translation layer. It also moves a whole class of defects from runtime surprise to build-time declaration — but only for shape. A document that validates perfectly and contains an invented invoice number is a contract success and a product failure, and keeping that distinction in view is most of the discipline here.

Key decisions

• Whether conformance is obtained by constraining generation or by checking afterwards. Constraining guarantees the shape and restricts the usable schema vocabulary; checking accepts any schema and leaves you handling failures.
• What happens on a violation: repair locally, return the error to the model and retry, fall back to a simpler shape, or fail the request. Repairing silently is the option that hides how often the contract is missed.
• How strict the schema is. Forbidding unknown fields catches drift and breaks the moment the model volunteers something useful; permitting them keeps things working and lets the real shape drift away from the declared one.
• How much semantic meaning is pushed into the schema — enumerations, patterns, numeric bounds — versus checked afterwards in code. Expressing it in the schema makes it enforceable at generation time if the mechanism supports that keyword, and silently decorative if it does not.
• Whether the response is consumed as it streams. Partial validation of an incomplete object requires a parser built for it, and a contract that can only be checked when complete removes the latency benefit of streaming.
• How the contract is versioned and how consumers learn it changed, given that the producer is a model whose output shape is determined by a prompt and a schema that version independently.
• Whether an optional field that is absent means the model had nothing to say, could not comply, or was never asked — three different outcomes that an absent key cannot distinguish.

Failure modes

• A repair step quietly fixes malformed output, nobody counts the repairs, and the true conformance rate is unknown until a model change pushes it past what repair can absorb.
• The schema uses keywords the constraining mechanism does not enforce — a string format, a conditional branch, a cross-field dependency — and they are dropped at registration, so the guarantee the team believes it has covers only part of the declaration.
• Required fields are added to the schema without being added to the instructions or the examples, so the model emits the key and fills it with a plausible value rather than a grounded one.
• Constrained decoding forces a syntactically valid object out of a model that had nothing to say, so "no answer" becomes a confidently populated structure instead of an empty result.
• The schema grows deeply nested to mirror an internal domain model, output quality falls, and the cause is a shape the model struggles to produce rather than the task.
• Free prose the user needs — the explanation, the caveat — is squeezed into one string field inside the object, and the product renders a wall of text that nothing in the contract governs.
• A streaming consumer renders a partially parsed object and the final chunk changes a field it had already displayed.
• The contract and the prompt that teaches the model to satisfy it live in different systems and are released separately, so for a window of time each is correct and together they are not.

Related capabilities

• Few-shot Examples — demonstrations are how a contract is taught, as opposed to enforced
• Prompt Templates — where the schema is rendered into the prompt alongside the instructions
• Instruction Hierarchy — the contract normally outranks a user's formatting preference and must say so
• Prompt Registry and Versioning — the schema and the prompt teaching it should version as one unit
• Tool Schema Validation — the same mechanics applied to arguments going the other way

Applies: Production readiness · Design questions