AI research and discussion on LessWrong. www.lesswrong.com
Distinguishes minimal (jagged, fallible) from maximal (idealised) superintelligence, arguing they differ in timelines, risks and mitigations, and that prosaic safety measures matter for the near-term minimal case.
A dialogue arguing that a superintelligence could overcome security, physical and containment limits, using side-channel attacks, self-replicating systems and persuasion as examples.
Conceptual analysis of why corrigible AI agents should prioritize a principal's newer commands over older ones, examining temporal authority, empowerment, and decision policies.
Argues that AI oversight is currently necessary despite AI moral status, and that utility-maximising coherence can make an AI's defection incentives worse than incoherence, so many AIs may not want to scheme.
A speculative essay sketching non-extinction ASI futures, arguing alignment may be intractable and that multi-agent instrumental convergence and monitoring problems could push a principal toward confining its agents.
Argues that superintelligent text-only AI persuasion is limited by I/O constraints, proposes levels of superpersuasion, and assesses the plausibility of each, concluding arbitrary behavioural control is unlikely.
A conceptual taxonomy comparing introspective and extrospective recursive self-improvement, arguing that the internal route offers fewer monitoring channels and weaker verification, with implications for safety.
Proposes training agents to have preferences only between outcomes with equal numbers of copies (POSC), so they neither pay costs to self-replicate nor are strongly disposed to it, as a safeguard against AI self-replication.