Context Engineering and Prompt Management
Few-shot Examples — Stack
Few shot examples › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
The choice is between a block that is the same for everyone and a block chosen per request, and everything else follows from it.
- Selection latency. What per-request selection costs before the model call begins, including an embedding of the input if one is needed.
- Cache behaviour. Whether the example block can sit in a stable leading region. A fixed block is cacheable; a selected one is not, and the difference is usually larger than the quality difference.
- Store governance. Where examples live, who may add one, and whether anything prevents raw production content from entering without redaction.
- Coupling to the contract. Whether a change to the output schema can mechanically flag the examples it invalidates, or whether that is remembered by a person.
- Set-level evaluation. Whether the mechanism supports measuring the block as a whole rather than justifying each example individually.
- Determinism. Whether the same input selects the same examples, which decides whether two requests are comparable and whether a past request can be replayed.
- Diversity control. Whether selection can avoid returning several near-identical demonstrations, which is the common failure of similarity-based selection.
- Token cost. The block's size relative to the window, and whether it is accounted for in capacity planning as a fixed occupant rather than treated as part of the prompt.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Fixed example block in the prompt artefact | Most production tasks; reviewable, cacheable, identical for every request | Cannot adapt to an unusual input, and grows by accretion until nobody can say what each example is for |
| Similarity-based selection from an example store (vector search over the input) | Tasks with many distinct sub-cases where one block cannot cover the space | Retrieves near-duplicates of the input, which invites copying; adds a retrieval dependency to the assembly path |
| Diversity-aware selectors (maximal marginal relevance over the example store) | Keeping a selected set varied rather than clustered around one neighbourhood | One more parameter to tune; can select a well-spread set that is less relevant than a focused one |
| Length-aware selectors | Products where the example block must fit a variable remainder of the window | Admitting examples by size systematically prefers short ones, which are often the least instructive |
| Prompt optimisers that choose demonstrations against a metric (DSPy bootstrapping and optimiser modules) | Teams with a scored development set and a willingness to let a compiler pick | The chosen set is a compilation artefact that reads oddly to humans and is only as good as the metric behind it |
| Curated example store with review (dataset tooling, annotation platforms, a reviewed repository folder) | Organisations where examples carry policy and need an owner and an audit trail | Process cost per example; the store ages, and nothing forces a periodic re-review |
| Fine-tuning instead of demonstrating | Very high volume with a stable task, where the example block dominates cost | Behaviour stops being legible and editable; every change becomes a training run, and the examples now live in a dataset version |
Models
Dynamic selection introduces a second model into the path, and its properties matter more than is usually assumed.
- Embedding models do the selection. What matters is whether their notion of similarity matches the product's notion of "a comparable case" — two support tickets about different products with identical phrasing are near neighbours in embedding space and are not useful demonstrations of each other.
- Dimension and latency set the cost floor, since the input must be embedded on every request before selection can run; a small model close to the application usually beats a larger hosted one here.
- Cross-encoder rerankers give better selection from a shortlist, at the price of a second inference pass before the main call, which is rarely justified for a handful of examples.
- Pin the embedding build. Re-embedding the example store with a different model while queries use the old one silently degrades selection, and the symptom is a gradual fall in answer quality with no change anywhere in the prompt.
- Example selection quality is bounded by the store, not by the embedder: a well-chosen nearest neighbour from a store with no relevant case is still not a demonstration of the task at hand.