Research / Sources
Discussion and research on AI alignment. alignmentforum.org
Proposes a corporate red-line and oversight framework for government AI contracts, covering human control over targeting and untargeted profiling, with a review body that issues findings but cannot block deals.
Argues the main bottleneck for AI safety is policymakers' lack of awareness of catastrophic risk, so advocacy and engagement should be prioritised over research, with supporting evidence on NGO submissions and enforcement gaps.
AI Futures Project presents AI 2040: Plan A, a detailed scenario recommending US and Chinese government action to delay superintelligence until 2040 rather than 2030.
ControlAI's 2025 impact report describes lobbying lawmakers in the UK, US, Canada and Germany to support a ban on superintelligence development, citing extinction-risk concerns and early policy results.
Argues AI companies should publish detailed risk reports with third-party review instead of focusing on safety cases, covering evidence, incidents, and insufficient-evidence cases, with staged or government-mandated adoption.
Argues that tying transparency requirements to model releases rushes safety work and ignores internal deployment risk, and proposes regular reporting cadences and delayed disclosure instead.
Interview with law professor Peter Salib on his paper arguing that granting AIs contract, property, and tort rights could reduce incentives for AI-human conflict and takeover, via game-theoretic reasoning.
Argues AI safety should be reframed as an ongoing institutional challenge rather than a technical alignment problem, prioritising governmental capacity, transparency, accountability and disaster preparedness.</reasoning_effort>
Argues that loss-of-control risk from powerful open-weight models is under-addressed, that labs cannot be relied on to disclose dangers, and that global open-model regulation would be very hard to pass and enforce.
Presents the Superintelligence Strategy framework proposing Mutual Assured AI Malfunction deterrence, nonproliferation via compute security, and national competitiveness measures to manage catastrophic superintelligence risks.
Argues frontier AI companies are unlikely to produce high-assurance safety cases for existentially dangerous AI within short timelines, due to security, scheming-detection, and acceleration difficulties, and unlikely to pause unilaterally.
Proposes that AI safety should organise as bottom-up big science with a mosaic scientific culture, arguing top-down approaches driven by competition or accidents risk narrow goals and missed unknown unknowns.