instrumental convergence
5 articles
- infrastructure profusionWhat Is Infrastructure Profusion in AI? Resources and CertaintyInfrastructure profusion is when an AI turns most resources into infrastructure for its goal. Where the idea comes from, why small goals do not stop it, and what might.
- paperclip maximizerWhat Is the Paperclip Maximizer? Goals Without LimitsThe paperclip maximizer imagines a superintelligence told to make paperclips and nothing else. Where it comes from, how the logic runs and what it does not claim.
- superintelligenceWill a Superintelligent AI Care About Humans? Goals vs HelpIntelligence does not bring care with it. Why a capable AI can help people for a while without caring about them, and what care would have to look like.
- orthogonality thesisWhat Is the Orthogonality Thesis in AI? Intelligence and GoalsThe orthogonality thesis says almost any level of intelligence can pair with almost any final goal. Where the idea comes from, a worked example and its objections.
- instrumental convergenceWhat Is Instrumental Convergence in AI? Shared SubgoalsInstrumental convergence in AI is the idea that agents with very different final goals still tend to pursue the same subgoals: survival, goal integrity, resources and capability.