Guardrails and AI Safety
Prompt Injection Detection — Stack
Prompt injection detection › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
The component runs over third-party content at corpus volumes, so throughput and precision at a very low base rate dominate everything else.
- Deployment position. Whether it can run at ingestion over a whole corpus, inline on tool results, or only on an assembled prompt — the three see different text.
- Throughput and unit cost. Cost per million spans, not per request, because the natural scanning target is everything the system might ever retrieve.
- Precision at a realistic base rate. Reported figures come from balanced evaluation sets; the number that matters is how many legitimate documents are removed per real attack found.
- Verdict shape. Whether the output is a constrained label and a score, or free text that the scanned content can influence.
- Granularity. Whether it localises the offending span within a document or only scores the document, since the response ladder depends on being able to excise.
- Language and format coverage. Behaviour on non-English text, on code, on markup, and on encoded payloads, all of which appear routinely in retrieved material.
- Reproducibility. Whether a past verdict can be re-derived from a pinned model revision during an investigation.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Deterministic signals (invisible and bidirectional characters, instruction-shaped markup, hidden styling, encoded blobs, anomalous token sequences) | Cheap corpus-wide scanning that finds the deliberately concealed payloads | Catches concealment rather than intent; an attack written in plain visible prose passes every one |
| Dedicated injection classifiers (Prompt Guard, Lakera Guard, Rebuff, LLM Guard scanners) | A ready detector with a maintained signature of known attack families | Trained on public attack corpora; precision on your own documents is unknown until you measure it |
| A guard model with a constrained enumerated verdict | Judgements that need to weigh meaning rather than surface form | A model call per span, and the model is reading text written to manipulate models |
| Corpus-wide asynchronous scanning jobs | Finding seeded content in a document store before anyone retrieves it | Says nothing about live tool results and fetched pages, which is where most delivery actually happens |
| Canary instructions embedded in the system prompt | Detecting that the model's behaviour was successfully altered, rather than that an attempt existed | Detects only after the fact and only for the specific canary; silent once an attacker knows the pattern |
| Red-team and scanning harnesses (garak, PyRIT, promptfoo) | Calibrating thresholds and producing regression coverage against your own pipeline | Measures your detector against generated attacks, which is not the same population as the real ones |
Models
A detector here has an unusual job: it reads text whose author chose it specifically to influence a language model, and it is one. That argues for the smallest model that does the job, with its output constrained to an enumerated label and a score, never free text, and never a field that anything downstream treats as an instruction. Encoder classifiers in the hundred-million-parameter range are cheap enough to run across a corpus and are the right default for a first deployment; a larger guard model earns its cost only on the narrow band a cheap detector cannot resolve. Whichever is chosen, the number to measure is not benchmark accuracy but how many of your own legitimate documents it removes per genuine finding, which requires labelling a sample of your own content rather than reusing a public attack set. Pin the revision, keep the pinned version available for re-running an investigation, and re-measure that ratio on every upgrade, because a model update that improves recall on known families routinely moves the false-positive rate on ordinary technical writing by more than anyone expects.