Prompt Injection Defense

What it is

Building context in a way that limits what text the system did not author can do once it is in front of a model. It covers where the trust boundary is drawn, how untrusted spans are marked and contained, how they are kept below instructions in authority, what is stripped or normalised before admission, and how much capability the request retains while such material is present. Classifying an incoming message as an attack, filtering what comes back out, and enforcing organisational policy at runtime are separate safety concerns; this capability is about the construction of the context itself.

Why it matters

Every mechanism in this domain assumes the text in the context means what its author intended. Indirect injection breaks that assumption at the source: a web page, a document, a calendar invitation, a code comment, or a tool result can carry text addressed to the model rather than to the reader, and a model that has been asked to read it will read all of it. The consequence is not a wrong answer but an unauthorised action, because the compromised context sits in the same request as the tools and the credentials. There is no known construction that makes a model reliably ignore instructions embedded in material it has been asked to process, so the design goal is to reduce what a successful injection can reach, not to prevent one.

Key decisions

• Where the trust boundary is drawn, and the honest answer for each producer. Anything that can be influenced by someone outside the team — including first-party stores containing user-submitted content — is on the untrusted side.
• How untrusted material is contained in the context: a distinct message role, a typed content block, or a delimiter. Only the first two are structural; a delimiter is a string the untrusted text can also contain.
• Whether the system's capability changes in the presence of untrusted content — a reduced tool set, read-only mode, or mandatory confirmation for anything with an effect. This is the control with the largest effect on blast radius and the largest cost to product flow.
• What is transformed on the way in: invisible and bidirectional control characters, markup that renders differently from its source, encoded payloads, and text styled to look like a system message.
• Whether a potentially malicious document is read in the main context at all, or in a separate call whose output is a narrow value.
• What happens to injected text once it is in the conversation record. A single poisoned tool result persists through history and can act on a later turn after everyone has moved on.
• How the defences are tested. Without an adversarial suite that is maintained, every control here is an assertion.

Failure modes

• Defence rests on a delimiter, and the untrusted document contains that delimiter followed by a convincing instruction, so the containment marks the end of the attacker's payload rather than the end of the data.
• The injection arrives through a tool result rather than through the user's message, so every check placed on user input never sees it.
• A model is asked to judge whether its own context contains an injection, using the same context that contains it, and the judgement is made by the compromised reasoning.
• The attack succeeds on turn two and executes on turn nine. The poisoned text persists in the conversation record, and the action is attributed to the unrelated request that happened to trigger it.
• Content is sanitised at render time for a user interface but admitted to the context in its raw form, so the model sees the hidden white-on-white instruction the reviewer never saw.
• Untrusted spans are correctly labelled, and a summarisation pass later rewrites them into the conversation's own narrative voice, laundering attacker text into apparently trusted history.
• The defence is a blocklist of phrases. It is bypassed by rewording, by another language, or by an encoding, and its existence stops anyone from reducing the agent's privileges.
• Capability reduction is implemented for the agent's own tool calls but not for a sub-agent it delegates to, which runs with the full tool set.

Related capabilities

• Context Isolation — the sub-call boundary that bounds what injected text can reach
• Instruction Hierarchy — the precedence rule injected text must sit beneath
• Context Provenance — supplies the trust label every control here depends on
• Dynamic Prompt Assembly — where containment and normalisation are applied
• Prompt Templates — the substitution boundary where external values first enter prompt text

Applies: Production readiness · Design questions