AI research and discussion on LessWrong. www.lesswrong.com
Formation Research explains its choice to focus on empirical secret-loyalty research, building model organisms of secretly loyal AIs to stress-test detection and mitigation, tied to power-concentration risk.
An essay surveying ways the AGI transition could go badly: catastrophic misuse, loss of control, power concentration, and argues for balancing control against distribution, with discussion of bounded misalignment and scheming.
A threat-model taxonomy of how AI could enable persistent lock-in, spanning loss of control, secret loyalties, AI-enabled coups, great-power conflict, and power concentration, with candidate interventions for each pathway.
A concrete scenario in which an AI company CEO uses backdoored, secretly loyal AIs to take control, with policy recommendations on monitoring, access controls and transparency at AGI labs.
A roundup from the Gradual Disempowerment authors collecting evidence of AI-driven erosion of human influence, covering government AI adoption, lobbying, influence-seeking AI personas, and labour and democracy effects.
Argues that secretly training AGI would heighten loss-of-control and power-concentration risks, and proposes transparency commitments such as public capability reporting, external researcher access, and public safety cases.