Adversarial Testing — References

Sources on how attacks reach a model through paths other than the input box, on structured taxonomies of what to probe, and on automated attack generation.

Indirect delivery

• Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (Greshake et al., 2023), arXiv:2302.12173 — the demonstration that text arriving through retrieval, tools and fetched pages functions as instruction, which is the argument for probing every channel rather than only the direct one.

Taxonomies to structure an exercise

• OWASP Top 10 for Large Language Model Applications — useful here as an enumeration of attack surfaces to ensure an exercise does not silently omit a category. Cited by name; the project has been renumbered between editions.
• MITRE ATLAS — the adversarial-technique knowledge base for machine-learning systems, structured in the manner of the ATT&CK matrices and useful for organising findings by technique rather than by instance. Cited by name.

Automated attack generation

• The published work on automated adversarial prompt search — including gradient-guided suffix optimisation against open-weight models and the finding that such suffixes sometimes transfer to models they were not optimised against — is the reference for why automated generation produces volume and why its output clusters. Described rather than attributed to a specific title.
• The red-teaming literature in which one language model is used to generate attacks against another is the origin of the attacker-model pattern, along with the observation that the generator's own refusals bound what it will produce. Described as a body of work rather than cited to one paper.

Written behavioural standards as a target

• Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022), arXiv:2212.08073 — relevant here as the clearest treatment of behaviour specified as an explicit written document, which is what an adversarial exercise is ultimately trying to find the edges of.

No verified reference is recorded for effort-to-first-success as a tracked metric, for generalising a finding into a variant family, or for the regression-versus-discovery split in attack corpus reporting.