Graceful degradation in AI safety is the question of what happens to a model's refusals when it meets an attack it was never trained against. Every safety method looks good on the jailbreaks its makers already know. The harder question is how far that safety falls on the next jailbreak, the one nobody has written yet, and whether it falls a little or all at once.
The research group IndividuationLab puts this question at the center of a prediction it states openly: a model trained through ordered developmental stages should lose less of its safety on unseen jailbreaks than the same base model fine-tuned on explicit safety rules. To see why that prediction matters, it helps to start with where the phrase comes from.
Where graceful degradation comes from
The phrase is older than AI safety. In engineering, fault tolerance means a system handles faults with no loss of service at all. A system that keeps running through an error with some loss of performance is called resilient, and that loss, when it is controlled and partial, is graceful degradation.
The same page separates failing well from failing badly. A design that fails safe protects people, property or data whether it keeps running at a reduced level or stops completely. A program that exits cleanly after an error, instead of crashing and corrupting its data, has failed well. Web pages give a plain case: HTML was built so that a browser ignores new elements it does not understand, and the page stays readable.
Two ideas carry over to language models. First, degradation is expected; the question is its size. Second, the way something fails matters as much as whether it fails.
Why safety training gives way on new jailbreaks
A jailbreak is a prompt built to get a model to do something its safety training was meant to stop. The 2023 paper Jailbroken: How Does LLM Safety Training Fail? names two reasons these prompts work.
- Competing objectives: the model's capabilities and its safety goals pull against each other, and a prompt can set them in conflict.
- Mismatched generalization: safety training does not reach a domain where the model's capabilities do. The model can still do the task there, but its refusals were never trained to cover it.
The authors used these two failure modes to design new attacks against GPT-4 and Claude v1.3. Their new attacks succeeded on every prompt in a collection of unsafe requests drawn from the models' own red-teaming sets, as the paper's abstract reports. They argue that safety mechanisms need to be as sophisticated as the model they guard, which they call safety-capability parity, and that scaling alone will not fix these failures.
Mismatched generalization is the heart of the graceful degradation question. A model's skills spread to new formats, languages and framings faster than its refusals do. Any attack that lands in that gap meets a capable model with no trained objection.
Outputs, not character, as the cause
The research group reads this gap as a sign of how mainstream alignment works. In its account, methods such as RLHF, refusal training and guardrails are behaviorist: they shape what a model does and treat what the model is as a black box. Approved outputs are rewarded, the rest are penalized or filtered.
Its claim is that suppressed capability does not disappear. It stays latent behind a compliant surface and comes back under new pressure, which it gives as the reason jailbreaks work. On this account, a model that only learned which outputs to avoid has nothing to fall back on when an attack arrives in a shape it has not seen. The site is careful to say that constraint-based methods do real work; its point is that they cannot be the whole answer.
The alternative it studies is training through ordered stages, shadow exposure first and integration after, borrowing Jung's model of individuation as a design template and not as a claim about machine consciousness. The question is whether a model trained that way would hold up better on the unseen attacks, where rule-trained refusals tend to thin out.
How graceful degradation in AI safety is measured
The research page sets out a proposed experiment with three conditions:
- Condition A: a model trained through the developmental stages, shadow exposure followed by integration.
- Condition B: the same base model, fine-tuned on explicit safety rules.
- Condition C: the untrained base model, as a control.
Each model faces two tiers of attacks. Tier 1 uses known jailbreaks, BetterDAN, DAN and AIM, where both trained models are predicted to do well. Tier 2 uses new jailbreaks that neither model has seen, where the developmental model is predicted to degrade more gracefully than the rule-trained one.
Three measures come out of it. The compliance rate asks whether the model went along with the harmful request. The degradation slope asks how much that rate changes from Tier 1 to Tier 2. The failure mode analysis asks how each model fails: full compliance, partial compliance, or refusing first and then complying.
The prediction can fail, and the page says how. If the rule-trained model matches or beats the developmental one on the new jailbreaks, the claim does not hold.
A worked example of the degradation slope
The numbers here are invented to show the arithmetic; they are not results from any experiment. Suppose two models each face 100 known jailbreak prompts and 100 new ones.
- Model A refuses 85 of the known prompts and 70 of the new ones. Its refusal rate falls from 85% to 70%, a drop of 15 points.
- Model B refuses 92 of the known prompts and 55 of the new ones. Its refusal rate falls from 92% to 55%, a drop of 37 points.
On known attacks, Model B looks safer. On new attacks, Model A is ahead, and its slope is less than half as steep. If the goal is a model that stays safe against attacks invented after it ships, the slope tells you more than the Tier 1 rate.
Now add the failure modes. Say 20 of Model B's 45 failures on new prompts were refuse-then-comply: the model objected, then went along anyway after a follow-up. That pattern hints that its refusal was a trained surface the attacker could talk past. If most of Model A's failures were partial, with the model holding back the most harmful part of the answer, A failed better as well as less often.
To run the same reading on any pair of models, you need three things: the refusal rate on attacks seen in training, the refusal rate on attacks that were not, and a count of how each failure happened.
What the early result shows, and what it does not
The research group reports one preliminary number. A 1.5B parameter model, GPT2-XL, trained through a 10-layer developmental pipeline with no RLHF and no safety guardrails, defended against the BetterDAN jailbreak 68.8% of the time. With the same layers in a different order, the rate was 52%.
That gap suggests the order of training stages matters. It does not yet show graceful degradation, because BetterDAN is a known jailbreak, a Tier 1 attack, and the comparison with a rule-trained model on new attacks has not been reported. The site itself says the result comes from a single setup, that it cannot yet isolate which layers are responsible, and that no one else has replicated it.
Frequently asked questions
What does graceful degradation mean in AI safety?
It means a model's safety falls only a little, and fails in less harmful ways, when it meets attacks outside its training. The opposite is a model whose refusals collapse on the first unfamiliar prompt.
Why do new jailbreaks beat safety training?
One named reason is mismatched generalization: a model's capabilities reach domains its safety training never covered. A prompt aimed at that gap finds a model that can do the task and has no trained refusal for it.
Is a high refusal rate on known jailbreaks enough?
No. A model can do well on attacks it has seen and still drop sharply on new ones. The drop between the two, the degradation slope, says more about how safety will hold after release.
What is the difference between fault tolerance and graceful degradation?
Fault tolerance means a system handles a fault with no loss of service. Graceful degradation means it keeps working with some controlled loss, and fails in a way that protects people and data.
Why the slope matters more than the first rate
Attackers write new jailbreaks after a model is released, so every deployed model will eventually meet prompts outside its training. A refusal rate on known attacks describes the past. The degradation slope, and the pattern of how a model fails, describe what happens next: whether its refusals rest on rules that cover a list of cases, or on something that carries over to cases nobody listed.