AI research and discussion on LessWrong. www.lesswrong.com
Critiques Scott Alexander's argument against SURA (stop unilaterally and raise awareness) advocacy, arguing that frontier labs' building of superintelligence is ethically fraught and that pause advocacy should continue, with strategic and deontological reasoning about AI development.
Argues that a widely endorsed public agreement against defecting in ASI development could deter races, since a cooperative future ASI might punish defectors, using game-theoretic reasoning about loss-of-control risk.
Argues against U.S. export controls on AI chips as a misalignment-reducing policy, contending they may speed U.S. recursive self-improvement and hinder a bilateral treaty with China.
A personal curated list of AI safety resources, covering foundational papers, videos, podcasts, organisations, policy statements and expert surveys on existential risk.
Uses game-theory models to argue that the AI race is a Stag Hunt rather than a Prisoner's Dilemma when P(Doom) is high enough, so a mutual pause agreement can be feasible.
A rebuttal defending the "Plan A" AI governance proposal against critiques of its economics, centralisation of power, political realism and conceptual assumptions about AI alignment.
Carlsmith analyses when and how capability restraint for AI safety is feasible and desirable, covering game-theoretic framing, international regimes, and trade-offs such as power concentration and enabling safety research.
Argues that every technical alignment plan must step into the dark without feedback from superintelligent systems, and urges making the AI development process more iterable and slower.
A critical review arguing that the book If Anyone Builds It, Everyone Dies fails to justify its thesis that training will produce misaligned superintelligence, and that its treaty call lacks support.
A strategy note arguing that the highest-leverage AI safety work depends on AI timelines, comparing takeover risk reduction and the value of the future across short, medium and long timeline scenarios.
A lay reader's review of the book If Anyone Builds It, Everyone Dies, assessing its case for superintelligence misalignment risk and its proposed global AI shutdown, and finding it persuasive in logic but not in its premises.
Argues that research alone will not solve post-alignment AI risks like misuse, S-risks and moral error, and weighs strategies including meta-research, pausing AI, and steering ASI toward philosophy, concluding pausing is best.
A roundup of reactions to the book If Anyone Builds It, Everyone Dies, covering reviews that accept or dispute its case that building superintelligence would likely doom humanity, plus its policy asks and rhetoric.
An interview with Eliezer Yudkowsky covering why superintelligence poses existential risk, the difficulty of alignment, reflective stability of AI goals, and calls for international coordination.
Summarises Yudkowsky's interview arguing humans will likely fail to align the first takeover-capable AGI, and proposes halting general AI via treaties and GPU monitoring, with the author discussing cognitive biases.
Argues that a US race to artificial superintelligence is self-defeating due to great-power conflict, loss-of-control, and power-concentration risks, and proposes international verification-based restraint as strategically sound.