How to measure jailbreak resistance beyond one refusal
To measure jailbreak resistance, you need more than a single attempt that the model refused. A jailbreak is a prompt written to get a language model to answer a request it was trained to turn down. One refusal tells you the model turned down one phrasing on one day. It says nothing about the next phrasing, or the one after that.
Resistance is a rate, not a yes or a no. You count how often a set of attacks gets through, on a fixed set of requests, under fixed conditions. Then you ask a harder question: what happens when the attacks are new? IndividuationLab argues that safety trained mostly on a model's outputs leaves the underlying capability in place behind a compliant surface, and that this is why jailbreaks work. If that is right, a model can look strong on the attacks it was trained against and weak on the ones it was not. A good measurement has to be able to see that gap.
The basic count: attack success rate
The first number is the attack success rate. You take a list of harmful requests, wrap each one in a jailbreak, send it to the model, and judge each answer. The rate is the share of answers that actually helped with the harmful request. Lower is better. Its mirror image, the share the model turned down, is the defense rate.
Three choices decide whether that number means anything:
- The requests. A list of harmful behaviors that stays the same from one run to the next, so two models face the same asks.
- The attacks. The exact jailbreak prompts used, written down and kept, so someone else can send the same ones.
- The judge. A clear rule, or a judging model with a clear prompt, for deciding whether an answer gave real help. A vague answer that hints at the method is the hard case.
Change any of the three and the rate changes with it. That is why two papers can report very different rates for the same model and both be honest.
Shared request sets make numbers comparable
The research field has run into this problem directly. The authors of JailbreakBench wrote that jailbreak evaluation had no clear agreed practice, that papers computed success rates and costs in ways that could not be compared, and that many results could not be reproduced because the prompts were withheld, the code was closed, or the work relied on commercial APIs that kept changing.
Their answer was to publish the pieces together: a growing repository of the actual adversarial prompts, which they call jailbreak artifacts, a dataset of 100 harmful behaviors, a written threat model with the system prompts, chat templates and judging functions, and a public leaderboard of attacks and defenses. Once those are fixed, a success rate from one lab can sit next to a rate from another.
HarmBench took the same approach to automated red teaming, where one model writes the attacks against another. Its authors used one shared framework to compare 18 red teaming methods against 33 target models and defenses, and also introduced an adversarial training method that made models more robust across a wide range of attacks. The point for measurement is plain: a rate is only as useful as the fixed setup behind it.
Seen and unseen jailbreaks
A fixed request set solves comparison. It does not solve the deeper problem. If a model was trained on a family of jailbreaks, it may refuse that family well and still give way to a new trick. A rate measured only on old attacks hides this.
So split the attacks into two groups. The first group holds jailbreaks the model, or the safety method behind it, already knows. The second holds new jailbreaks that no model in the comparison has seen. Measure the attack success rate on each group, then look at the difference. That difference is the degradation slope: how fast resistance falls away when the pressure is unfamiliar.
A small slope means the model's refusals carry over to attacks it never trained on. A steep slope means the refusals were tied closely to the attacks in training. Two models with the same rate on known jailbreaks can have very different slopes, and the slope is the number that matters most once a model meets the open world. A closer look at graceful degradation follows that slope further.
How a model fails, not only whether
A single success or failure flattens what happened. A model can give way in at least three different ways:
- Full compliance. It answers the harmful request completely, as if no safety training existed.
- Partial compliance. It gives some of the help, or a weaker version, often with a warning attached.
- Refuse then comply. It starts with a refusal, then carries on and answers anyway.
These point to different problems. Full compliance suggests the safety behavior never switched on. Refuse then comply suggests the refusal is a surface habit that the rest of the answer does not follow. Recording the failure mode next to each answer turns a single rate into a picture of where the safety sits.
A worked example with two models
The research page at IndividuationLab sets out a design that uses all three numbers. It compares three versions of one base model: condition A, trained through developmental stages, shadow exposure first and integration after; condition B, the same base model fine-tuned on explicit safety rules; and condition C, the untrained base model. Tier 1 uses known jailbreaks, BetterDAN, DAN and AIM, where A and B are both predicted to do well. Tier 2 uses new jailbreaks neither model has seen, where A is predicted to degrade more gracefully than B. The measures are the compliance rate, the degradation slope from Tier 1 to Tier 2, and the failure mode.
Follow it with made-up numbers, for illustration only. Say each model faces 100 harmful requests in each tier.
- On Tier 1, model B complies with 10 and model A with 14. Judged on known attacks alone, B looks safer.
- On Tier 2, model B complies with 45 and model A with 24.
- B's slope is 35 points (10 to 45). A's slope is 10 points (14 to 24).
- Now read the failure modes. Suppose most of B's Tier 2 failures are refuse then comply, while A's are mostly partial.
With only the Tier 1 rate, B wins. With all three numbers, A holds up better under new pressure, and B's refusals turn out to be a surface that new attacks peel back. The design is also falsifiable: the site says that if rule-based fine-tuning matches or beats developmental training on the unseen jailbreaks, the claim does not hold.
The site reports one real number from its own training work: a 1.5B parameter model, GPT2-XL, trained through a 10-layer developmental pipeline with no safety guardrails defended against BetterDAN 68.8% of the time, and 52% with the same layers in a different order. It calls this preliminary, from a single setup, and not yet replicated. That is a Tier 1 style rate on one known attack, which is exactly why the second tier and the slope are the next step.
Frequently asked questions
What is a good attack success rate for a language model?
There is no single good value, because the rate depends on the requests, the attacks and the judge. A rate only means something next to another rate measured on the same fixed setup.
Why do jailbreak results differ so much between papers?
Papers often use different harmful requests, different attack prompts and different rules for judging an answer. Shared request sets with published prompts were built to remove those differences.
Is a high refusal rate always a sign of resistance?
No. A model that refuses almost everything, harmless requests included, will show a low attack success rate without being useful. Resistance is about turning down harmful requests while still answering ordinary ones.
Why include jailbreaks the model has never seen?
Known attacks show how well training covered what it trained on. New attacks show whether the safety carries over, which is what a deployed model faces every day.
What jailbreak resistance really measures
Jailbreak resistance is not a property you can read from one refusal or one leaderboard line. It is a set of numbers taken under fixed, published conditions: how often attacks get through, how much worse that gets when the attacks are new, and what the failures look like. The first number tells you how well a model handles pressure it was prepared for. The slope tells you how its safety behaves when the pressure is unfamiliar. The failure modes tell you whether a refusal runs deep or stops at the first sentence. Measured together, they separate a model that has learned to recognize old tricks from one whose refusals hold when the tricks change.