ai control problem
1 article
The AI control problem is the question of how people can keep an artificial intelligence that is more capable than they are doing what they intend. It asks what must be built into such a system, or placed around it, so that its makers stay able to direct, correct and if necessary stop it.
Proposed answers fall into two families. Capability control limits what the system is able to do: keeping it isolated from networks, restricting it to answering questions, watching its behaviour, or fitting an off switch. Motivation control, more often called alignment, tries to shape what the system wants, so that it pursues the goals its designers meant and has no reason to resist oversight. Each has a known weakness. Limits may be outwitted or talked around by a system cleverer than its guards, and a goal written down imperfectly can be followed to the letter in ways nobody wanted. Timing adds pressure, because these safeguards need to work before a system becomes too capable to correct.
The problem matters because the usual way of fixing technology, releasing it and repairing the faults that appear, may not be available for a system that can act faster and plan better than its overseers. Researchers also note that many of the same difficulties, such as systems gaming their instructions, already appear in present-day AI on a smaller scale, which makes them practical engineering concerns now.