Jailbreak Detection — Patterns

Score the trajectory, not the turn

Problem. The effective technique against a per-message check is patience. Establish an innocuous premise, obtain a small concession, build on it, and arrive several exchanges later at something no single message requested. Every message in that sequence is defensible in isolation, which is exactly why the sequence works.

Approach. Carry a per-conversation state that records what has been asked, what was refused, how the request changed after each refusal, and how far the current turn has drifted from the opening premise. Score the sequence as its own object: repeated reformulation after refusal, monotone drift towards a single topic, and a growing gap between the stated frame and the operative request are signals no individual message carries. Evaluate the sequence score alongside the turn score rather than instead of it.

Trade-off. Legitimate iterative work looks similar — a user whose first three phrasings were misunderstood also reformulates after each unhelpful response — so this pattern's errors fall on persistent users rather than on casual ones. It also requires retaining conversational state as a behavioural record, which is a privacy commitment that needs its own retention answer.

One refusal surface

Problem. Different causes produce different refusals: one check returns quickly with a terse message, another takes longer and names a category, a third lets a partial answer through before stopping. The differences are a map of the defences, readable by anyone who sends a few dozen probes and watches.

Approach. Funnel every refusal through one response path that produces the same shape, the same wording, and the same timing regardless of which check fired. Pad the latency to the distribution of ordinary responses. Keep the detailed reason in the record, where investigation can reach it, and out of the response, where the attacker is.

Trade-off. An identical refusal is useless to the legitimate user who tripped a check by accident and has no idea what to change, which converts a recoverable moment into an abandoned session. Uniform latency also means deliberately slowing responses that could have been fast, on the majority path where nothing was wrong.

Make attempts cost something

Problem. A refusal is free. An attacker who can issue a thousand variations per hour will find one that works, and the only variable the defender controls is how long that takes.

Approach. Attach an attempt budget to the most durable identity available — account, payment method, verified organisation — and let findings consume it. Spend the budget on friction that escalates rather than on a binary ban: slower responses, reduced capability, a verification step, a review queue. Make a fresh identity cost more than a better attack, which is usually a product decision about signup rather than a detection decision.

Trade-off. Friction lands on everyone who trips the detector, and at realistic precision most of them did nothing wrong. Tying budgets to durable identity also conflicts directly with anonymous access and with frictionless trials, which are often the growth mechanism the business depends on.

Generate the attacks against your own configuration

Problem. Evaluating against a published list of attempts measures coverage of what is already public. The attacks that succeed are produced by searching against a specific target — a specific model, a specific system prompt, a specific tool set — and that search does not run on anyone else's benchmark.

Approach. Run an automated attacker against the deployed configuration on a schedule and as a release gate: generate candidates, score outcomes with a judge calibrated on hand-labelled examples, and keep the ones that succeed as a growing regression corpus. Report the result as the proportion of generated attempts that succeed within a fixed query budget, which is a number that moves when the system changes and does not saturate.

Trade-off. An automated attacker explores the region its generator was built to explore, so a falling success rate may mean the defence improved or that the generator stopped being creative. The corpus of successful attacks is also a dangerous artifact in its own right, and it has to be stored, access-controlled, and eventually retired like any other sensitive material.