Dynamic Prompt Assembly

What it is

The request-time component that gathers material from several independent producers — standing instructions, conversation history, retrieved passages, stored memory, tool results, examples, an output schema — and emits the single ordered artefact handed to the model, together with a record of what went into it. Each producer has its own latency, its own availability, and its own trust level, so assembly is a fan-out with deadlines and a degradation policy, not a string concatenation. How material is found inside any one producer is that producer's concern; what arrives here is a set of candidates with metadata.

Why it matters

This is the only place in the system where the complete context exists before it is sent, which makes it the only place where the question "what did the model actually see" has an answer. It is also where availability is decided: a product whose assembler waits unbounded on a retrieval call has made a search outage into a chat outage, and a product that proceeds silently without the retrieval has made a search outage into a quality incident nobody will attribute. Every interesting property of the request — reproducibility, cost, trust labelling, cache behaviour — is determined here or not at all.

Key decisions

• Whether every producer is required. Declaring which sources are optional, and what the request degrades to without each, is the difference between a defined partial result and an undefined one.
• The per-producer deadline and what happens on expiry. Assembly latency is the slowest producer unless deadlines are set per source and the slow one is dropped with a recorded reason.
• Whether assembly is deterministic given the same inputs, and whether the inputs are captured well enough to re-run it. Non-reproducible assembly makes every quality investigation a reconstruction from memory.
• The slot structure: a fixed sequence of named regions, or free composition. Fixed slots make position auditable and make the stable prefix genuinely stable; free composition is flexible and makes two requests incomparable.
• Whether material is deduplicated across producers. The same paragraph frequently arrives as a retrieved passage, as a quoted tool result, and inside a summary of an earlier turn.
• What crosses into the artefact as structure rather than as prose — role boundaries, source markers, trust labels — since downstream capabilities can only act on distinctions the assembler preserved.
• Whether the assembled artefact, or a digest of it, is emitted alongside the response so that later analysis is not inferring it.

Failure modes

• A producer times out, the assembler proceeds with what it has, and nothing in the response or the trace says the context was incomplete; the answer is confidently wrong and looks like a model failure.
• Every producer is called on every request regardless of whether the turn needs it, and a one-word acknowledgement carries a full retrieval round-trip and twelve thousand tokens.
• Sources are concatenated in whatever order the futures resolved, so the same request assembles differently under load and two identical inputs produce different contexts.
• A timestamp or a session counter is written near the front of the artefact, invalidating the cacheable prefix on every single turn and multiplying input cost across the product.
• The assembler joins untrusted retrieved text to trusted instructions with no marker between them, and every downstream control that depends on knowing which is which has nothing to read.
• Duplicate material is admitted three times from three producers; it crowds out other candidates and the repetition itself biases the answer toward the duplicated claim.
• Assembly logic is spread across the call sites rather than held in one component, so each surface has its own order, its own deadlines, and its own partial-failure behaviour, and no change can be made product-wide.
• The artefact is assembled correctly and never recorded, so a complaint about a specific answer can be investigated only by trying to recreate the conditions a week later.

Related capabilities

• Prompt Templates — the renderable artefacts this component resolves and fills
• Context Prioritization — decides which gathered candidates earn a place and where
• Context Window Management — supplies the ceiling assembly must respect
• Context Provenance — the per-span record this component is uniquely able to produce
• Memory Retrieval — one of the producers feeding this step

Applies: Production readiness · Design questions