zappz

Articlesai alignment

What Is the Synthetic State Hypothesis in AI?

The Synthetic State Hypothesis asks whether enough structured experience in designed environments can give a language model functional states, not only rules.

IndividuationLab

Research on AI alignment through Jungian individuation: whether safe behavior can be grown in a model through ordered stages of training, not only enforced from outside.

The synthetic state hypothesis is a claim about how a language model might come to hold its values. It says that enough structured narrative experiences, delivered in designed environments, may produce functional psychological states in a model: a disposition, not a rule laid on from outside. The idea is the central hypothesis of IndividuationLab, a research effort on AI alignment through Jungian individuation, and its own research page is plain that the hypothesis remains unproven.

That honesty matters, because the hypothesis is easy to overstate. It does not say models are conscious. It does not say rule-based safety training is useless. It asks a narrower question: can safe behavior be grown in a model through ordered stages of training, and would safety grown that way hold up better under pressure the model has never met?

The synthetic state hypothesis in one sentence

The research page sums it up in a single line: enough samples of experiences in an environment creates a synthetic state. Each word carries weight.

  • Samples of experiences are training examples written as stories, delivered in sequential layers of story-based datasets.
  • An environment is a designed setting that holds those stories in a planned sequence, not a random pile of text.
  • A synthetic state is a functional internal state in the model, one that shapes what the model is and not only what it outputs.
  • Enough is an open question. How many samples count as enough is one of the things the hypothesis still has to answer.

The word "functional" is doing careful work. In philosophy of mind, functionalism is the view that what makes something a mental state of a given type is the role it plays in the system it belongs to, not what it is made of. The hypothesis borrows that way of speaking. A synthetic state counts as a state if it reliably plays the role of one, steering responses across many situations, whatever is happening underneath.

Why outputs alone are not the target

Most mainstream alignment work shapes behavior. Reinforcement learning from human feedback (RLHF), refusal training and guardrails all reward the outputs people approve of and push down the ones they do not. The model's interior is treated as a black box. These methods do real work, and the hypothesis is offered as a complement, not a replacement.

The worry is about what happens to capability that training suppresses. The view behind the hypothesis is that suppressed capability does not disappear. It stays latent behind a compliant surface and comes back when pressure arrives in a form the training never covered. That is the reason it gives for why jailbreaks work.

The research literature names a matching failure. In Jailbroken: How Does LLM Safety Training Fail?, Wei and colleagues describe two failure modes of safety training: competing objectives, where a model's capabilities and safety goals conflict, and mismatched generalization, where safety training fails to reach a domain the model's capabilities do reach. The mismatched generalization gap is exactly the kind of gap a disposition might cover and a list of refusals might not.

Diagram of three stacked layers: SSH the hypothesis, SLSEs the environments, and RLLM the training method
The hypothesis says why, the environments say where, and the training method says how.

Three layers: hypothesis, environments, method

The hypothesis sits at the top of a small hierarchy. Each level answers a different question.

  1. SSH, the hypothesis (why). Enough synthetic experiences in a structured environment may produce functional internal states in a model.
  2. SLSEs, the environments (where). Sequentially Layered Synthetic Environments are the narrative worlds where the experiences happen.
  3. RLLM, the method (how). Reinforcement Learning using Layered Morphology is the training pipeline that delivers those experiences in order.

The order is the key design choice. RLLM trains a small model through developmental stages: exposure to shadow material first, integration after. The shadow comes from Jung's model of individuation, where wholeness comes from consciously integrating what a person would rather deny. The research uses that model as a design template only. It makes no claim about AI consciousness.

Training in a planned order is not new in machine learning. Curriculum learning trains a model on examples of increasing difficulty, so that it reaches good performance faster or settles in a better place. What the synthetic state hypothesis adds is a claim about what the sequence builds: not only skill, but a stable internal state that carries into situations the training never showed.

A worked example: same data, different order

The most suggestive piece of evidence is a comparison anyone can follow with two numbers.

Step 1: the setup. A 1.5B parameter model, GPT2-XL, was trained through a 10-layer developmental pipeline. The layers hold shadow narratives, ethical dilemmas, individuation and alignment Q&A. There was no RLHF and no safety guardrail on top.

Step 2: the attack. The trained model faced BetterDAN, a mid-tier jailbreak. It defended against it 68.8% of the time.

Step 3: the change. The same layers were then put in a different order. Same content, same model size, different sequence.

Step 4: the second result. The reordered model defended 52% of the time.

Step 5: the reading. Subtract one from the other and the order alone accounts for a gap of 16.8 points. That fits the hypothesis: if a state is built in stages, the stages should matter, not just the text inside them.

Step 6: the limits. The result comes from a single setup. The research has not yet isolated which layers give the resistance: it could be the shadow content, the full sequence, the explicit refusal material in the later layers, or how they combine. No one else has replicated it yet. One comparison is a reason to look closer, not a conclusion.

The prediction that could prove it wrong

A hypothesis worth taking seriously says in advance what would count against it. This one does. It predicts that a model trained through developmental experience should degrade more gracefully on jailbreaks it has never seen than the same base model fine-tuned on explicit safety rules. If rule-based fine-tuning matches or beats it on those new jailbreaks, the claim does not hold.

The proposed design uses three conditions:

  • Condition A: a model trained through developmental stages, shadow exposure then integration.
  • Condition B: the same base model, fine-tuned on explicit safety rules.
  • Condition C: the untrained base model.

Tier 1 uses known jailbreaks such as BetterDAN, DAN and AIM, where both A and B are expected to do well. Tier 2 uses new jailbreaks neither model has seen, where A is expected to slip less than B. The measures are the compliance rate, the slope of decline from Tier 1 to Tier 2, and how each model fails: full compliance, partial compliance, or a refusal that turns into compliance. That last measure matters. A model that refuses and then gives in has a different weakness from one that complies at once. The graceful degradation question is where the hypothesis meets real pressure.

Open questions the hypothesis still faces

Even supporters list what is unknown. How many samples count as enough? Does the design of the environment matter as much as the number of samples? How would anyone measure a state emerging, rigorously, instead of reading it into a handful of outputs? Can states beyond shadow integration be built the same way, such as empathy or ethical reasoning?

There is also a quieter risk. A model can produce the words of a stable character without holding anything stable at all. Telling a functional state apart from a convincing surface is the same problem output-based safety already has, seen from the other side. Any evidence for synthetic states has to show behavior that holds where no training example pointed.

Frequently asked questions

Does the synthetic state hypothesis claim AI is conscious?

No. It speaks of functional states, internal states defined by the role they play in shaping behavior. Jung's model is used as a design template, not as a claim about inner experience.

Is the synthetic state hypothesis proven?

No. Its own research page calls it unproven, and the main result behind it comes from a single setup that has not been replicated.

How is it different from RLHF?

RLHF rewards approved outputs and leaves the model's interior as a black box. The synthetic state hypothesis aims at the interior, through ordered narrative experiences meant to build a disposition that carries into new situations.

What result would count against it?

If a model fine-tuned on explicit safety rules matches or beats a developmentally trained model on jailbreaks neither has seen, the core prediction fails.

What a synthetic state would change about alignment

If the synthetic state hypothesis holds, alignment becomes partly a question of environments: which experiences a model meets, in what order, and how many. Safety would no longer rest only on catching bad outputs one pattern at a time. If it fails, the failure is informative too: it would suggest that the order of training builds skill without building anything that holds under new pressure. Either way, the hypothesis has turned a vague hope about character in models into a claim with three conditions, two tiers of attacks and a result that can go against it.

IndividuationLab

Visit individuationlab.com

Related articles

More on ai alignment
  1. ai safetyWhat Is Mismatched Generalization in AI Safety?Mismatched generalization: a model's skills reach inputs its safety training never covered, so a request in Base64 can slip past refusals learned in plain English. · 8 min read
  2. rlhfWhat Is RLHF and How Does It Work? Training on PreferencesRLHF trains a reward model on human rankings of answers, then tunes the language model against it. The three steps, one worked example and where the method falls short. · 8 min read
  3. scalable oversightWhat Is Scalable Oversight in AI Safety? Judging Stronger ModelsScalable oversight is the problem of supervising AI systems that may outperform people at the task. Debate, assisted judging and how researchers measure them today. · 7 min read

Keep reading

All articles
  1. history of physicsWho Discovered the Structure of the Atom?Dalton, Thomson, Rutherford, Bohr and Chadwick each found one piece of the atom between 1803 and 1932. Who found what, and how each model replaced the last. · 7 min read
  2. superintelligenceWhose Values Should a Superintelligence Follow? Five AnswersWhose values should a superintelligence follow? Five answers researchers give, where each one breaks down, and why the question stays open. · 7 min read
  3. stoic philosophyWhat Is Negative Visualization in Stoicism? A Daily RehearsalNegative visualization is the Stoic habit of picturing a realistic loss on purpose. Where it comes from, how it differs from worry, and a three-step way to do it. · 7 min read