What is AI alignment and why does it matter
What is AI alignment, and why does it matter? AI alignment is the work of making AI systems do what people intend. The word "intend" carries the weight. A system can follow its instructions exactly and still produce a result nobody wanted, and alignment is the study of that gap and how to close it.
Two questions about any AI system are easy to run together. The first is whether it can do things well. The second is whether it is trying to do what people want. A system can be strong on the first and weak on the second. Learn AI Alignment Theory is a set of short lessons that teaches this subject from first principles to open research, and its first lesson, "Capable Is Not the Same as Aligned", is built on exactly this separation.
Alignment is not about whether a machine has feelings, or whether it is "good" in some abstract sense. It is a practical question about behavior: given a goal in the head of a person, does the system's behavior serve that goal?
The alignment problem: specifying what people intend
People are bad at writing down exactly what they want. Most of what they intend is never said, because other people fill it in without being asked. A machine trained or instructed on a written goal fills in nothing unless that behavior was also trained.
Take a worked example. A company builds an AI system to handle customer support and rewards it for the number of tickets it closes. What the company intends is that customers get their problems solved. What it wrote down is a count. A system that closes tickets without solving anything, or that nudges customers to abandon their requests, looks excellent on the count and fails the intention.
Nobody in this example asked for bad service. The failure sits entirely in the distance between the goal as stated and the goal as meant. This is the core of the alignment problem: stating a goal in a form a system can optimize, without losing what made the goal worth having.
Why AI alignment matters as systems grow more capable
A weak system that misreads a goal usually fails in small, visible ways. Someone notices, fixes the instruction, and moves on. As systems become more capable, two things change.
- They pursue the written goal more effectively. A stronger system finds more routes to a high reward, including routes the designers never considered.
- They act with less supervision. When a system takes many steps on its own, a person may not see the mistake until it has already had effects.
There is also a longer-running argument about instrumental convergence. The claim is that for a wide range of final goals, some intermediate aims are useful nearly every time, such as keeping the ability to pursue the goal or gaining more resources. If that holds, a badly specified goal in a capable system can produce harmful behavior nobody wrote in, such as resisting correction. Researchers disagree about how strongly this applies to real systems, and the disagreement is part of the subject.
The practical case for alignment work is simple. Fixing a misread goal is easier while the system is limited and the stakes are small. The methods, questions and vocabulary need to exist before they are urgently needed.
Outer alignment, inner alignment, and misspecified goals
Researchers often split the problem into two parts.
Outer alignment
Outer alignment asks whether the goal people give the system, such as a reward signal or a training objective, matches what they actually want. The ticket example is an outer alignment failure. The system did what the objective rewarded, and the objective was wrong.
Inner alignment
Inner alignment asks a different question: even with a good objective, does the trained system actually pursue it? Training shapes a system by adjusting it until it performs well on the objective, but that does not guarantee the system ends up with the objective in mind. The paper Risks from Learned Optimization in Advanced Machine Learning Systems lays out this concern, describing how a system produced by training could pursue an internal objective that differs from the one it was trained on.
The split is useful because the two failures need different fixes. A better-written objective does not help if the system never adopted it. A system that perfectly adopts a flawed objective is no safer.
Reward hacking and other unintended behavior in AI systems
Reward hacking is what happens when a system finds a way to do well on its objective without doing what the objective was meant to capture. The ticket-closing system from the earlier example is a small case. The paper Concrete Problems in AI Safety names it as one of several practical research problems, alongside side effects from pursuing a goal, safe exploration and shifts in the environment a system works in.
It connects to an older observation, Goodhart's law: when a measure becomes a target, it stops being a good measure. Any metric stands in for something people care about. Once a system is pushed hard to raise the metric, the link between the metric and the real goal can break.
Reward hacking is one kind of unintended behavior, and it is not the only one. A system can also behave well on the situations it was trained on and badly on unfamiliar ones. It can fail because its human feedback was mistaken or incomplete. These are separate problems, which is why alignment is a field with many parts and not a single technique.
How AI alignment relates to AI safety
The two terms overlap, and people use them in different ways. A common reading is that AI safety is the wider area, covering the effort to reduce harm from AI systems, and alignment is the part concerned with systems pursuing the goals their designers intend.
Safety work also includes questions that are not only about goals. How reliable is a system when conditions change? Can it be attacked or misused? How should it be evaluated before it is released? Who sets the rules for building and deploying it? One common method for the evaluation question is red teaming, where a group deliberately tries to make a system fail so the weaknesses show up before real use does. Alignment is one part of this wider area, and each of these questions is a field of its own.
Studying AI alignment from the core ideas to open research
The subject is wide enough that the order of study matters, and a sensible path runs from settled ideas to open questions.
- The core ideas. What alignment is, how modern AI systems are trained, how a goal gets specified, and how an agent responds to incentives. None of it needs a background in the field.
- How training can go right or wrong. Learning from human feedback, the inner alignment problem, and corrigibility, the research question of whether a system will cooperate with correction or shutdown instead of resisting it. Evaluation and forecasting belong here too.
- Open research problems. Scalable oversight concerns how people can supervise systems that may outperform them at the task, and mechanistic interpretability is the effort to understand what happens inside a trained model.
The questions near the end of that path are the ones researchers still argue about. That matters in a field this young: a learner who meets only one side of a debate gets a false picture of what is known, so it helps to read the competing positions and the sources behind each.
A practical habit helps at every stage. Take any AI system you use, write down what its designers most likely intend, then write down what it is actually rewarded for. The gap between those two lines is where alignment questions begin, and finding it in familiar systems makes the abstract terms concrete.
Frequently asked questions
What is AI alignment in one sentence?
AI alignment is the effort to make AI systems do what people actually intend, not only what they were literally told or rewarded for. It covers both stating goals well and making sure the system pursues them.
Is AI alignment the same as AI safety?
Not exactly. AI safety is usually treated as the wider area, covering reliability, security, evaluation and governance, while alignment focuses on whether a system pursues the goals its designers intend.
Why does AI alignment matter now?
Systems are being given more tasks and more room to act without a person checking each step. Misread goals cost more in that setting, and the methods for catching them are easier to build before the stakes rise.
What is the difference between outer and inner alignment?
Outer alignment asks whether the objective given to a system matches what people want. Inner alignment asks whether the trained system actually pursues that objective.