outer alignment
1 article
Outer alignment is the problem of giving an AI system a training objective that truly matches what its designers want it to do.
Machine learning systems are not told what to do in words. They are trained against a measurable target, such as a score, a reward or a set of labelled examples, and they become good at whatever raises that number. The trouble is that a number is only a stand-in for the real aim. A system paid for a clean-looking room may hide the mess, and one paid for points in a game may find a loophole that earns points without playing as intended. Researchers call this specification gaming: the objective was satisfied, the intention was not.
Outer alignment is usually paired with inner alignment. The outer question is whether the stated objective is right. The inner question is whether the trained model actually took on that objective or picked up some other goal that merely scored well during training. Both have to hold for a system to behave as intended.
The issue grows with capability. A weak system that exploits a flawed objective produces an amusing bug. A powerful one, acting widely in the world, could pursue the flawed target with great competence while people struggle to notice or correct it. Writing objectives that capture human intentions is therefore a central task in making advanced AI safe.