What AI safety and character means
AI safety and character is the question of whether a language model can be safe because of what it is, and not only because its outputs are policed. Most safety work today acts on the outputs. A model is rewarded for approved answers, penalized for the rest, and wrapped in filters. That makes a model that behaves well on the requests its trainers foresaw. It does not settle whether anything inside the model changed.
IndividuationLab, which researches AI alignment through Jungian individuation, puts the problem sharply. It argues that RLHF (reinforcement learning from human feedback, which trains a reward model on human preferences and then trains the model against it), refusal training and guardrails shape what a model does while treating what the model is as a black box. Suppressed capability, on that account, does not disappear. It stays latent behind a compliant surface and comes back under new pressure. The site does not ask anyone to drop those methods; it says they do real work but cannot be the whole answer.
So the real question is narrower than it first sounds. Nobody doubts that training changes a model. The question is whether training can change it at the level where a refusal stops being a rule the model obeys and becomes a disposition it has.
Character in the old sense of the word
Philosophers have argued about character for a long time, and virtue ethics gives the clearest vocabulary for it. The Stanford Encyclopedia of Philosophy entry on virtue ethics describes a virtue as an excellent trait of character: a disposition, well entrenched in the person who has it, that goes all the way down. It sets this apart from a mere habit, such as being a tea drinker. A virtue shapes what a person notices, expects, values, feels, desires and chooses, not only what they end up doing.
The same entry draws a line that matters a great deal for AI. It separates full virtue from continence, or strength of will. The fully virtuous person does the right thing without a struggle against contrary desires. The continent person does the right thing too, but only by controlling a desire or a temptation to do otherwise.
From the outside, the two can look identical. Both refuse the bribe. The difference shows up when the pressure changes shape, gets stronger, or arrives in a form the person never rehearsed. The continent person is only as reliable as the control holding the desire back.
Why output training builds continence
Training on outputs, what is sometimes called behaviorism in AI alignment, is in these terms a way of building continence. The underlying ability to produce a harmful answer is learned in pretraining. Safety training then teaches the model to hold that ability back on the kinds of request it was shown. The capability is still there; what changed is the control around it.
Research on jailbreaks fits this picture. In Jailbroken: How Does LLM Safety Training Fail?, Wei and colleagues propose two failure modes of safety training. Competing objectives arise when a model's capabilities and its safety goals pull against each other. Mismatched generalization happens when safety training fails to reach a domain where the model's capabilities do reach. Both are failures of control, not of the capability itself, which is exactly what a continent agent would show.
This is why a refusal can hold on a plain request and give way when the same request is wrapped in a role play, an unusual format, or a long fictional setup. The training taught the model to recognize and resist a certain surface. A new surface finds the capability without passing through the control.
What training for character would have to do
If continence is the problem, character is the target, and the hard part is saying what that would mean for a model. IndividuationLab borrows a template from Carl Jung's idea of individuation: wholeness through consciously integrating the shadow rather than denying it. It uses this as a design template, not as a claim that models are conscious.
In its training experiments, small models are trained through ordered developmental stages. Exposure to the shadow comes first, and integration comes after. The models are then attacked with jailbreaks. The idea is that a model which has met the harmful material and been trained to place it, instead of only being trained to refuse it, might hold a disposition that does not depend on recognizing a surface.
The site reports one preliminary result. A 1.5 billion parameter model, GPT2-XL, trained through a 10-layer developmental pipeline with no RLHF and no safety guardrails, defended against a mid-tier jailbreak called BetterDAN 68.8% of the time. With the same layers in a different order, the figure fell to 52%. The site is plain that this comes from a single setup, that it cannot yet say which layers are responsible, and that it has not been replicated. What it does suggest is that the order of training mattered, which is what a developmental account would expect and a pure rule account would not.
A worked example with three models
A claim about character is only useful if something could show it false. The research page sets out a design for doing that, and it is easy to follow step by step.
- Start with one base model. Every condition uses the same model, so any difference comes from training.
- Make three versions. Version A is trained through developmental experience, shadow exposure then integration. Version B is the same base model fine-tuned on explicit safety rules. Version C is the base model left untrained, as a control.
- Attack them with known jailbreaks. On attacks of the kind the rules were written for, version B may do as well as A, or better. That would say little about character, because continence works on rehearsed temptations.
- Attack them with jailbreaks none of them has seen. This is where the two accounts part. If A has a disposition and B only has controls, A should degrade more gracefully as the attacks get stranger.
- Read the result honestly. If B matches or beats A on the unseen attacks, the claim does not hold, and the site says so in advance.
The useful thing about this design is that it turns a philosophical distinction into a prediction. Continence and character look the same on familiar ground. They are supposed to come apart on unfamiliar ground, and that is the ground the experiment is built around.
What would count as a yes
A single good result on one jailbreak would not show that safety lives in a model's character. Several things would have to line up. The advantage would need to show on attacks the model was never shaped against. It would need to survive other model sizes and architectures. And it would need to be replicated by people who did not design the training.
There is also a quieter sign worth watching. A model with character, in the virtue sense, should be safe in a way that does not cost it much. The continent agent spends effort holding the desire back; the virtuous one does not. For a model, that might show as refusals that stay steady across phrasings, without a growing pile of special cases bolted on after each new attack.
Frequently asked questions
Is a jailbreak proof that a model has no character?
Not on its own. People with real virtues still fail under enough pressure. A jailbreak shows that a refusal depended on recognizing a surface when the same request in a new form gets through.
What is the difference between continence and virtue?
Both lead to the right action. The virtuous person acts well without struggling against a contrary desire, while the continent person acts well by holding that desire back.
Does training for character mean dropping safety rules and guardrails?
No. The argument is that constraints do real work but are not enough on their own, especially against attacks nobody foresaw.
Does a model need to be conscious to have character?
The question here is about dispositions, which can be measured by behavior under new pressure. Jung's model is borrowed as a template for training, not as a claim that the model has an inner life.
Whether character can be trained into AI
The honest answer today is that nobody knows yet. Output training builds something close to continence: a capability held back by controls that work where they were aimed and slip where they were not. Character would be a disposition that holds where nobody aimed it. Virtue ethics has long said the two look alike until circumstances change, and jailbreaks are those changed circumstances. Whether training can build the second kind of model is now a question with a design that could answer it, a first small result that leans one way, and the replications that would settle it still to come.