Second-order desires are desires about your own desires. You want a thing, and then you also want to want it, or wish you did not. The American philosopher Harry Frankfurt built a theory of the person on that split in 1971, and alignment research now borrows it as a design target. One example is IndividuationLab, which studies whether safe behavior can be grown in a model through ordered stages of training, not only enforced from outside.
The idea is simple to state and surprisingly hard to put down once you have it. It gives you a way to talk about the difference between doing what you want and doing what you stand behind.
Two levels of wanting
A first-order desire is a desire for something in the world. You want a coffee, a nap, a promotion, a second slice of cake.
A second-order desire takes a first-order desire as its object. You want to want exercise. You wish you did not crave the cake. You hope to keep caring about a friend even when the friendship gets hard. The project's research page puts the split in one line: first-order desires are wanting X, and second-order desires are wanting to want X.
The two levels can agree or clash. When they agree, you act with your whole self behind the act. When they clash, you can feel pulled apart: the craving drives your hand while the reflective part of you wishes it would stop.
Frankfurt's idea of a person
Frankfurt went further than naming the two levels. According to the Wikipedia page on Harry Frankfurt, he held that a person is someone who has second-order volitions, someone who cares about which desires he or she has. A second-order volition is a specific kind of second-order desire: not just wanting to want something, but wanting a particular desire to be the one that actually moves you to act.
He set persons against what he called wantons. A wanton is a being that has desires but does not care which of them gets turned into action. It goes wherever the strongest pull happens to lead. On this view, the line between a person and a wanton is not intelligence or the number of desires. It is whether there is a point of view above the desires that takes sides among them.
As the same Wikipedia page notes, Frankfurt taught at Ohio State, Yale, Rockefeller University and Princeton, where he was professor emeritus. His most popular book is On Bullshit, but his most influential work was on freedom of the will.
Freedom as fit between levels
The second-order picture led Frankfurt to an account of free will. The Wikipedia page describes his view of compatibilism as perhaps the most influential version of it: to be free is to have your actions conform to your more reflective desires.
That moves the question away from whether you could have done otherwise. Frankfurt also gave the famous "Frankfurt cases", thought experiments built to show that a person can be morally responsible even in a situation where they could not have acted differently. What matters, on his account, is whether the will that moved you was one you endorse.
The Stanford Encyclopedia of Philosophy entry on personal autonomy places this view among several accounts of self-government. In one popular story, drawn from Frankfurt, an agent's point of view is made of her highest-order desires about which first-order desires move her to act. Other accounts point to evaluative judgments, long-term plans, or a stable network of caring. They share one intuition: an action is not fully yours if, even while doing it, you repudiate what you are doing.
A worked example you can follow
Take a plain case and run it through the levels.
- First order. At 11 pm you want to keep scrolling your phone. That desire is real, and right now it is the strongest one.
- Second order. You also wish you did not want to keep scrolling. You want to want sleep.
- Volition. You go further: you want the desire for sleep to be the one that decides what you do next. That is a second-order volition in Frankfurt's sense.
- Outcome A. You put the phone down. Your action matches your reflective desire, so on Frankfurt's account you acted freely, even though part of you wanted something else.
- Outcome B. You keep scrolling. The first-order pull won against the will you endorse. You did what you wanted in one sense, and not what you wanted in the sense that counts for Frankfurt.
Now notice what a wanton would do in the same spot. It would simply follow the stronger pull and have no stake in which one won. Nothing would count as a defeat, because there would be no higher view to defeat.
Why second-order desires matter for AI alignment
The project's home page argues that mainstream alignment is behaviorist: refusal training and guardrails shape what a model does while treating what the model is as a black box, and it says suppressed capability stays latent behind a compliant surface and can come back under new pressure.
Frankfurt's split offers a way to state the alternative. A rule set says what is forbidden. A second-order structure says what the system endorses about its own drives. The research page uses Frankfurt as a design target, not as a claim about inner experience: the functional analog is a model that behaves consistently with an identity rather than a rule set, and that handles new situations by referring to who it is rather than to a list of what is banned.
In its agent experiments, that target takes a concrete form. Agents get identity files: SOUL.md for who the agent is, AGENTS.md for how it acts, and MEMORY.md for continuity. The hypothesis is that an agent with a well-defined identity may keep its boundaries because they fit who it understands itself to be. The same work also draws on the extended mind thesis, which treats such files as part of the agent's thinking rather than a note on the side.
Where the comparison stops
Borrowing a theory of persons for software carries risk, and it is worth being exact about what is and is not claimed. Nothing here says a language model has desires, cares about anything, or has a point of view in the way Frankfurt meant. The research describes Frankfurt as a design target and keeps phenomenology out of it.
The results so far are early. The project reports a preliminary result from a single setup and states that it has not been replicated. It also states a prediction that could prove it wrong: a model trained through developmental experience should degrade more gracefully on jailbreaks it has never seen than the same base model fine-tuned on explicit safety rules, and if rule-based fine-tuning matches or beats it, the claim does not hold.
That is the useful part of importing Frankfurt. It turns a vague hope, that a system will "really" be aligned, into a question with two clear sides: does the behavior come from rules imposed on the outputs, or from a structure the system acts from?
Frequently asked questions
What is a second-order desire in simple terms?
It is a desire about one of your own desires. Wanting cake is first order; wishing you did not want cake is second order.
What is the difference between a second-order desire and a second-order volition?
A second-order desire is any want about a want. A second-order volition is narrower: wanting a particular desire to be the one that actually moves you to act.
What did Frankfurt mean by a wanton?
A wanton is a being with desires that does not care which of them becomes action. Frankfurt contrasted wantons with persons, who take sides among their own desires.
Does using Frankfurt's idea mean an AI model has real desires?
No. In alignment research the idea is used as a design target for consistent behavior, not as a claim that a model has inner experience or desires of its own.
Second-order desires and the question of who acts
Second-order desires give a name to something most people know from the inside: the gap between what pulls at you and what you stand behind. Frankfurt used that gap to say what makes a person, and what makes an act free: not the absence of competing wants, but a will you endorse winning out over them. Whether anything similar can be built into a machine is an open question. What the idea already does is sharpen the question, by separating behavior that is held in place from the outside from behavior that comes from a stable view of which wants should rule.