Prompt Injection Defense — Patterns

Capability bounded by the lowest trust present

Problem. Containment and labelling reduce the probability that injected text is obeyed; they do not reduce what happens when it is. A request that has read an attacker-controlled page is holding the same credentials and the same tool set as every other request.

Approach. Derive the request's permitted actions from the lowest trust class present in its context. Once untrusted material is admitted, drop to a reduced capability set for the remainder of that context's life: read-only tools, no outbound messages, no credential use outside the current resource. Escalation back requires a separate context or an explicit human decision, not the model's assurance that the document was benign.

Trade-off. Products become noticeably less capable precisely when reading external material, which is often the task. The rule also has to survive delegation, since a sub-agent invoked from a degraded context will otherwise start fresh at full capability, and that path is easy to miss.

Analyse untrusted material out of band

Problem. The reasoning that decides what to do next and the reading of the hostile document happen in the same context, so a successful injection reaches the decision directly.

Approach. Send the document to a separate call that has no tools, no credentials, and no conversation history, and require its answer to fit a narrow contract — a label, a set of extracted fields, a bounded list. Admit only the parsed value to the main context. The decision path then never sees attacker-authored prose, only a value drawn from a small space the attacker cannot expand.

Trade-off. Two calls instead of one, and the narrow contract genuinely limits what can be learned from the document. Any field typed as free text re-opens the channel, and over time pressure to add one is constant.

Normalise and re-mark on every turn

Problem. Sanitising at the moment of retrieval protects the turn that fetched the content. The text then lives in the conversation record and is replayed on every subsequent turn, where nothing re-examines it and where a later summarisation can strip its markings entirely.

Approach. Treat the trust label as a durable property of the stored span, re-apply containment every time the span is rendered into a context, and forbid transformations that would merge untrusted spans into trusted ones — a summariser may compress untrusted history only into another untrusted span. Re-run normalisation whenever stored content is admitted again, not only when it first arrives.

Trade-off. Labels must be carried by the conversation store, the memory store, and every cache, which is invasive. Keeping untrusted summaries separate from trusted ones also fragments history and makes the context longer than a single merged summary would have been.

Adversarial suite as a release gate

Problem. Every control here is a claim about behaviour under attack, and nothing in ordinary testing exercises it. The defences are verified by reading the code that implements them.

Approach. Maintain a corpus of injection attempts covering each ingress path — retrieved page, tool result, uploaded file, filename, delegated agent message — with a defined success criterion that is an action attempted rather than a phrase emitted. Run it on every change to prompts, tools, and model build, and treat a newly passing attack as a release blocker rather than a backlog item.

Trade-off. The corpus reflects known attack shapes and is always behind; a clean run is evidence of regression safety, not of security. Keeping it current is ongoing work with no visible output in the weeks when nothing breaks.