Dynamic Prompt Assembly — Patterns

Required and optional producers with declared degradation

Problem. When a source is slow or down, the assembler either waits, fails the request, or quietly continues without it. The third is almost always what happens in practice, and it is the one with no visible signal.

Approach. Classify each producer as required or optional at design time, and for each optional one write down what the request becomes without it — a narrower answer, a disclaimed answer, or a refusal. On expiry or error, drop the producer, attach a structured note to the assembly record, and let the response surface the degradation where the user would otherwise be misled. Required producers fail the request explicitly.

Trade-off. Enumerating degradations is real design work for every combination, and most teams only write the single-failure cases. Surfacing degradation to users costs trust in the moment and buys it back over time, which is a product decision rather than a technical one.

Fixed slot order with volatile material last

Problem. Composition order that follows completion order produces an artefact that differs between two runs of the same request, breaks every prefix cache, and makes position an accident.

Approach. Define the artefact as an ordered list of named slots — instructions, schema, examples, stable task framing, retrieved material, recent turns, the current input — and fill slots as producers complete, emitting only when all deadlines have passed. Keep everything that changes per turn strictly behind everything that does not, and forbid writing anything volatile into the leading slots.

Trade-off. A fixed order cannot be the best order for every request, and the constraint that nothing volatile may precede stable material rules out some legitimate designs, such as putting the user's question first so the model reads the evidence with the question in mind.

Assembly manifest emitted with the request

Problem. After the fact, nobody can say which passages were present, which producer contributed them, what was dropped, or why the answer cited something that is not in the index today.

Approach. Build a manifest as a by-product of assembly: for each admitted span, its producer, its source identifier, its token count, and its position; for each rejected or expired producer, the reason. Emit the manifest with the response and keep it on the same identifier as the trace. Keep the full text only on a sample and on flagged requests; keep the manifest always.

Trade-off. The manifest is small but constant overhead on every request, and it is only useful if something queries it. It also records source identifiers, which is a disclosure surface of its own when the manifest is exposed to clients rather than kept internal.

Cross-producer deduplication before admission

Problem. The same content legitimately arrives from several producers at once — as a retrieved chunk, as a quotation inside an earlier assistant turn, and inside a rolling summary. Admitting all three wastes budget and makes a single claim look corroborated.

Approach. Normalise candidates to a comparable form, detect near-duplicates by a cheap textual signature before any expensive scoring, and keep one copy — preferring the instance with the strongest provenance and the fullest surrounding context. Record the suppressed duplicates in the manifest rather than discarding them silently.

Trade-off. Near-duplicate detection on text that differs only in a number or a negation will collapse two genuinely different statements into one, which is worse than the duplication it prevents. Keeping the instance with the best provenance also systematically discards the freshest copy when the freshest copy came from the least traceable producer.