Research / Sources
Discussion and research on AI alignment. alignmentforum.org
Forecasters update their timelines using METR's estimate of AI-driven R&D acceleration, predict a Superhuman AI Researcher around 2027, and argue this makes pausing frontier development urgent.
Argues that rapid military integration of AI, including classified deployment with little oversight, lowers the capabilities a misaligned AI needs for takeover, and recommends visibility and governance measures.
AI Futures Project presents AI 2040: Plan A, a detailed scenario recommending US and Chinese government action to delay superintelligence until 2040 rather than 2030.
Argues that controllable superintelligence research would inevitably yield unsafe ASI along the same technical path, so safe ASI projects require a global ban on ASI development with enforcement first.
ControlAI's 2025 impact report describes lobbying lawmakers in the UK, US, Canada and Germany to support a ban on superintelligence development, citing extinction-risk concerns and early policy results.
Lays out a framework of Plans A to E describing how different levels of political will shape the best strategy for handling misalignment risk, with estimated probabilities and takeover risks for each.
Speculative economic analysis of whether the leading AI country could outgrow the rest of the world via superexponential growth, giving it a decisive strategic advantage without military action, and how technological diffusion and trade affect this.
Analyzes failure modes of instruction-following as an AGI alignment target, including shutdown resistance, principal authentication and jailbreaking, human-controlled proliferation, and conflicts from mixed training targets.
A forecasting scenario of AI progress toward superintelligence, covering misalignment risks such as sycophancy and deceptive demos, self-replication concerns, security races, and the limits of interpretability.
Carlsmith sets out a conceptual framework for reaching alignment victory or costly non-failure, built on safety progress, risk evaluation and capability restraint, with waystations such as global pause and automated alignment research.
Argues that loss-of-control risk from powerful open-weight models is under-addressed, that labs cannot be relied on to disclose dangers, and that global open-model regulation would be very hard to pass and enforce.
A blog post on scenario forecasting for AI futures, arguing for framing debates around the share of stable positive versus negative outcomes and sketching world-model variables and crossroads; largely conceptual and forecasting methodology.
Presents the Superintelligence Strategy framework proposing Mutual Assured AI Malfunction deterrence, nonproliferation via compute security, and national competitiveness measures to manage catastrophic superintelligence risks.
A personal scenario analysis arguing AI catastrophe is more likely through messy race dynamics, institutional unpreparedness and coordination failure than through a single clear misalignment event.
A strategic playbook for a hypothetical frontier developer on minimizing extreme AI takeover risks, covering goals, outcomes, and prioritization heuristics for scaling, safety preparation, and governance.
Proposes that AI safety should organise as bottom-up big science with a mosaic scientific culture, arguing top-down approaches driven by competition or accidents risk narrow goals and missed unknown unknowns.