Argues that autonomous AI researchers will overwhelm peer review, funding and human understanding of AI safety science, and proposes a metascience organisation to address these failures.
A distillation of the AI2040 alignment roadmap, laying out a Plan C strategy for a responsible frontier lab: phases of mitigation, pause, handoff to AIs at a Min-H capability threshold, and alternative exit plans.
Forecasts that AI will automate capabilities research before safety research, argues this ordering is risky, and proposes differential-acceleration steps such as building safety-research automation tools and granting safety researchers early model access.
Proposes criteria for choosing which alignment research directions to hand off to automated AI researchers, covering slop risk, parallelizability, sub-problem length, progress toward ASI alignment, and legibility to labs.
Argues that AI companies' plans to bootstrap alignment using weakly superhuman AI are dangerously overconfident, and that they should openly acknowledge the risk and push for global coordination.
Argues that AI companies are unlikely to actually devote human-level AI systems to alignment research under alignment bootstrapping plans, since they would prioritize capabilities as they do with human researchers.
A short introductory course on AGI alignment covering misalignment risk arguments, specification gaming and goal misgeneralization, DeepMind's amplified oversight approach, and institutional safety practices.