Structured Output Contracts — Patterns

Shape for the consumer, not for the domain

Problem. The schema is generated from an internal domain model because that is convenient, so the model is asked to produce deep nesting, polymorphic unions, and fields whose names mean something only inside the codebase. Conformance falls and the failures look like model incompetence.

Approach. Design the response shape as a flat, shallow structure with descriptive field names and small enumerations, covering exactly what the consumer needs, and translate into the domain model afterwards in code. Where a union is unavoidable, make the discriminator an explicit enumerated field rather than something inferred from which keys are present.

Trade-off. A translation layer is code to maintain and a second place where the shape is defined, and a flattened response can lose relationships the domain model expressed naturally. Products with very large response objects end up with a translation step substantial enough to need tests of its own.

Escape hatch inside the contract

Problem. A schema that only expresses success forces the model to produce a well-formed answer in cases where the honest answer is that the input did not contain one. Constrained decoding makes this worse, because a conforming object will be produced no matter what.

Approach. Make refusal and uncertainty part of the contract: a status field with explicit values for a complete answer, a partial one, and an inability to answer, with the payload required only in the first case and a reason required in the others. Instructions and examples have to teach the negative branch as deliberately as the positive one, or it will never be chosen.

Trade-off. An available escape hatch gets used, sometimes for inputs that were answerable, so the refusal rate becomes a metric that needs its own threshold and review. It also adds a branch to every consumer, and consumers that ignore the status field are worse off than if it did not exist.

Conformance measured before repair

Problem. Repair and retry make the contract appear perfectly satisfied from outside. The proportion of generations that were right first time is invisible, so a slow drift goes unnoticed until repair stops being enough.

Approach. Record the outcome of the first generation attempt separately from the outcome delivered to the caller: valid, repaired, retried and then valid, or failed. Alert on movement in the first-attempt rate rather than on the delivered rate, and sample failing generations with their rendered prompt so the cause can be read rather than guessed.

Trade-off. It is pure instrumentation: it finds nothing by itself and depends on someone watching a number that is healthy almost all the time. Keeping failing generations also means retaining raw model output, which carries whatever was in the request.

Contract and teaching released together

Problem. The schema sits in the application and the prompt that teaches the model to satisfy it sits in a prompt store, with separate release paths. A field is added to one and reaches production hours before the other.

Approach. Bind the schema, the instructions that describe it, and the examples that demonstrate it into a single versioned unit that is released atomically, and make the runtime refuse to pair a schema version with an instruction version it was not released with. Where the two genuinely must ship separately, require every change to be compatible in both directions for the length of the window between releases.

Trade-off. Atomic release couples a prompt edit to an application deploy, which removes the main benefit of keeping prompts outside the build. The compatible-both-ways alternative is cheaper and relies on every author remembering a discipline that only matters during a window nobody can see.