Few-shot Examples

What it is

Worked instances of the task placed in the context so the model infers from them what a good answer looks like. Each example pairs an input with the response the product would have wanted, and the set as a whole communicates things prose struggles with: the level of detail, the handling of edge cases, the voice, the point at which the right answer is to decline. The examples may be a fixed block authored once or a set selected per request from a larger store.

Why it matters

Some behaviours are far cheaper to show than to describe. A paragraph trying to specify how terse an answer should be is longer, more ambiguous, and less effective than two examples of the right length. Demonstrations are also the main lever available when a model is nearly right — the shape is correct and the register is wrong — where editing instructions tends to produce large, unpredictable swings. The cost is that the example block is usually the largest static occupant of the context, and that everything it contains is copied with a fidelity that includes its mistakes.

Key decisions

• Whether the set is fixed or selected per request. A fixed block is cacheable, reviewable, and identical for everyone; dynamic selection fits the input better and makes every request's context different, which complicates comparison and cache economics alike.
• How many to include. The gain flattens quickly for most tasks while the cost does not, and the right number is a measurement for this task and this model rather than a convention.
• Where the examples come from. Real production cases carry the distribution and the data classification of production traffic; written examples are safe and tend to be cleaner and more uniform than reality.
• How the label or outcome distribution is balanced. A set drawn from real traffic inherits its skew, and a model shown nine approvals and one rejection learns a prior about approvals.
• Whether examples demonstrate failure and refusal as well as success. A set of only clean successes teaches that an answer is always available.
• How they stay synchronised with everything else they encode. Every example embeds the output contract, the tone rules, and the policy in force when it was written, so each of those changes silently invalidates the set.
• Whether the set is evaluated as a unit. Examples are usually added one at a time to fix a specific case and almost never removed, and their collective effect is rarely measured.

Why it is not the output contract

A schema says what is structurally permitted; demonstrations say what is typical, and they shape everything the schema leaves open — length, hedging, how an ambiguous input is treated, which of several valid structures gets used. They also fail differently. A contract violation is detected mechanically and can be retried; an example-induced problem produces perfectly conforming output that is subtly wrong in the same way every time, and is found only by reading outputs. Products usually need both: the contract to make the result parseable, the examples to make it good. See Structured Output Contracts for the enforcement side.

Failure modes

• The model copies surface features of the demonstrations. An entity name, a city, or a phrase from an example appears in outputs for unrelated inputs, and it is reported as a hallucination.
• One example contains a small error — a mislabelled case, a wrong unit, an inconsistent date format — and the model reproduces it faithfully on every request for as long as it is there.
• The example set is drawn from production and nobody redacts it, so customer names and identifiers are shipped in the prompt of every request including requests from other customers.
• The label distribution in the set is skewed and becomes the model's prior, so a classifier's measured bias tracks the example block rather than the input.
• Dynamic selection retrieves an example nearly identical to the current input, and the model copies its answer rather than deriving one.
• Examples are added over months to fix individual complaints until the block is the largest part of the context and several of its members now contradict each other.
• The output contract changes and the examples are not updated, so the model is shown one shape and instructed to produce another, and it produces something between the two.
• Dynamic selection is added to a previously static block, the cached prefix stops matching, and input cost rises across the product with no corresponding change in quality.

Related capabilities

• Structured Output Contracts — the enforced shape examples demonstrate
• System and Developer Instructions — the described behaviour examples complement
• Context Prioritization — examples compete with evidence for the same room
• Prompt Registry and Versioning — the example set versions with the prompt it belongs to
• Prompt A/B Testing — how a change to the set is justified rather than assumed

Applies: Production readiness · Design questions