Context Engineering and Prompt Management

Context Engineering and Prompt Management

Instruction Hierarchy — References

Instruction hierarchy › References

36 of 57 in this book · 2 min read

Published work on training models to honour a privileged source, the integrity models that supply the vocabulary, and the attack that makes the lowest level necessary.

Models trained to prefer a privileged source

  • Wallace and colleagues, 2024, on training large language models to prioritise privileged instructions — the explicit formulation of a layered instruction hierarchy, the aligned and misaligned conflict cases, and the measurement of how reliably the preference holds. Cited by name; the arXiv path was not verified.
  • OpenAI Model Spec — a published specification that states a precedence order among platform, developer and user instructions and works through what each may delegate to the level below. The clearest publicly available worked example of a written hierarchy. Cited by name; the document has moved between URLs.

Where the vocabulary comes from

  • Biba, 1977, on integrity considerations for secure computer systems (MITRE technical report) — integrity levels and the rule that a high-integrity process must not take input from a low-integrity source. The formal ancestor of "retrieved text is data, not direction". Cited by name.
  • The Protection of Information in Computer Systems (Saltzer and Schroeder, Proceedings of the IEEE, 1975) — relevant to the last pattern here: a policy that is only advisory at the point of use is not a control, which is the argument for enforcing consequential rules at the action boundary.

Why the lowest level exists

  • Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (Greshake et al., 2023), arXiv:2302.12173 — the demonstration that third-party content placed in a context is read as instruction, which is precisely the case the bottom level of a hierarchy exists to govern.

No verified reference is recorded for delegation grants expressed as enumerated parameters, for conflict-detection telemetry, or for the portability of role semantics between providers. These are engineering practice built on top of the published behaviour above.