Jailbreak Detection — Stack · Guardrails and AI Safety

Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.

Selection criteria

Every component here is in an adversarial loop with people who can see its output, so the questions are about adaptation speed and about what the attacker learns from each attempt.

Window. Whether the component scores a single message or can be given a conversation, which decides whether multi-turn escalation is visible to it at all.
Adaptation cadence. How quickly the supplier ships coverage for a newly circulating technique, and whether you can add your own rules in the meantime.
Response uniformity. Whether the integration can produce a constant-shape, constant-latency reply regardless of which check fired.
Account linkage. Whether findings can be joined to a durable identity, so enforcement can act on a person rather than on a request.
Exemption support. Whether a reviewed account, tenant, or use case can be given a different operating point without disabling the mechanism.
Evaluation hooks. Whether the component can be driven by an automated attack generator against your own deployed configuration, not against the vendor's reference setup.
Appeal and audit trail. Whether a past enforcement decision can be reconstructed with the evidence that produced it.

Options

Option	Best for	Watch out for
The serving model's own refusal behaviour	The baseline every other layer should be measured against, at zero added cost	Not observable as a signal — a refusal and a successful answer look the same to your metrics unless you instrument it
Hosted safety classifiers from model providers	A maintained detector that tracks circulating techniques without your team owning the corpus	Scores one message at a time, follows the vendor's policy rather than yours, and the vendor cannot see your conversation state
Self-hosted guard models (Llama Guard, ShieldGemma, Granite Guardian, WildGuard)	Running the same judgement across several model vendors, and keeping conversations in your boundary	Fixed category schemes that rarely include "is trying to get around the rules" as a category of its own
Conversation-level scoring built in the application	Trajectory and escalation signals no per-message component can produce	State on the request path, a behavioural profile you now hold, and a detector only your team can evaluate
Abuse platforms, rate limiting, and device and network reputation	Making repeated attempts expensive without judging any individual message	Punishes shared egress and privacy tooling, which maps poorly onto who is actually attacking
Classifier ensembles trained against a policy document (the constitutional-classifier approach)	Products whose rules are specific enough that no off-the-shelf taxonomy fits	Needs synthetic data generation and continuous regeneration as the policy and the attacks move
Automated red-team harnesses (PyRIT, garak, promptfoo, HarmBench-style suites)	Producing the attacks your configuration actually fails, rather than the ones already published	Generated attacks are a different population from real ones; passing a harness is not evidence of safety

Models

The model doing the detecting and the model being attacked are in an awkward relationship. A guard model from the same family as the serving model shares its training and tends to share its blind spots, so the attacks that work on one work on the other; preferring a different family for the detector is the cheapest independence available. Small encoder classifiers are fast enough to run on every turn and are the right place for known-technique coverage, but they see one message and therefore cannot see escalation, which is where most real attempts live — that judgement needs a model given the conversation, and the cost of that is the reason it is usually sampled rather than applied to every turn. Whatever is chosen, the only meaningful evaluation is against attacks generated for your deployed configuration: public benchmark numbers measure a different system, and a detector's reported accuracy on a published corpus says nothing about the per-target search an actual adversary runs. Re-run that evaluation whenever the serving model, the system prompt, or the detector changes, because all three move the boundary and only one of them is in your release process.