Guardrails and AI Safety
Action Risk Classification — References
Action risk classification › References
Hazard analysis as an engineering discipline, the defence-in-depth model that explains why a single valuation is never the whole control, and the risk taxonomies an organisation is likely to already use.
Hazard analysis
- Engineering a Safer World: Systems Thinking Applied to Safety — Nancy G. Leveson, MIT Press,
- The argument that hazards arise from unsafe control actions in a system rather than from component failures, which is why valuing a tool rather than a call, in a context, misses most of what matters. Cited by name.
- Normal Accidents: Living with High-Risk Technologies — Charles Perrow, Basic Books, 1984. Tight coupling and interactive complexity as the properties that make consequences disproportionate to the triggering action, and the reason reach is not the same dimension as severity. Cited by name.
Why one valuation is not the control
- Human Error — James Reason, Cambridge University Press, 1990, and the accompanying account in Human error: models and management, BMJ, 2000. The layered-defences model in which no single barrier is expected to hold, which is the framing for a value that feeds several independent consumers rather than one gate. Cited by name.
Risk vocabularies
- Open FAIR — The Open Group's Risk Taxonomy and Risk Analysis standards. A decomposition of risk into frequency and magnitude with defined sub-factors, useful mainly as a discipline for naming dimensions explicitly rather than blending them into a single adjective. Cited by name.
- NIST AI Risk Management Framework — cited here for its characteristics of trustworthy systems, which are the categories an auditor will expect a risk scale to map onto even though they are framed for an organisation rather than for a request.
No verified reference is recorded for per-call risk valuation in agent systems, for anchoring risk levels to organisation-specific worked examples, or for back-testing a risk scale against reversal and incident history. The safety-engineering literature analyses hazards in designed systems and assumes the action set is fixed, which is the assumption an agent breaks.