Guardrails and AI Safety

Guardrails and AI Safety

Prompt Injection Detection — References

Prompt injection detection › References

16 of 57 in this book · 2 min read

The paper that established the delivery model, early documentation of the technique, the one large study of attacks produced by real people, and the standing argument about detection in security.

The attack and its delivery

  • Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (Greshake et al., 2023), arXiv:2302.12173 — cited here for the delivery inventory rather than the defences: the paper's enumeration of where a payload can be planted is the checklist against which the coverage of a scanner should be audited.
  • Ignore Previous Prompt: Attack Techniques For Language Models — Fábio Perez and Ian Ribeiro,
    1. The early documentation of goal hijacking and instruction leaking as distinct techniques, useful because the two leave different traces and a detector tuned for one will not see the other. Cited by name.
  • Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game — Sam Toyer and colleagues, 2023. A large corpus of attacks and defences written by human players rather than generated, which is the closest available approximation to what people actually try. Cited by name.

Why detection alone is not the control

  • Outside the Closed World: On Using Machine Learning for Network Intrusion Detection — Robin Sommer and Vern Paxson, IEEE Symposium on Security and Privacy, 2010. The argument that classification works poorly for finding attacks compared with finding anomalies in well-defined normal behaviour, and that the cost of an error in a security setting is far higher than in the settings these methods were developed for. Directly transferable, and the most useful corrective to optimistic detector benchmarks. Cited by name.
  • OWASP Top 10 for Large Language Model Applications — cited here for the explicit position that injection has no complete mitigation, which is the premise behind treating detection as a reporting and triage layer rather than as a barrier. Cited by name.

No verified reference is recorded for response ladders keyed to detector confidence, for source-level flag aggregation, or for measured precision of injection classifiers at production base rates. Vendors publish benchmark figures on balanced sets; nobody has published the ratio that matters.