AI research and discussion on LessWrong. www.lesswrong.com
Critiques Scott Alexander's argument against SURA (stop unilaterally and raise awareness) advocacy, arguing that frontier labs' building of superintelligence is ethically fraught and that pause advocacy should continue, with strategic and deontological reasoning about AI development.
Surveys competing definitions of recursive self-improvement in frontier AI and argues that a ban is hard to operationalise, so labs should sharply limit AI authority over successor models.
Argues that capabilities research can sometimes improve safety by reshaping the safety-usefulness Pareto frontier, while most capabilities research pushes developers toward less safety, with examples from RLVR, inference and burning the lead.
A dialogue defending Substrate Needs Convergence, the claim that AGI alignment is impossible because evolutionary pressures override engineered goals, and debating its political and philosophical implications for AI x-risk policy.
Speculates that AI could progressively take over organizations, from unintentional takeover to explicit handoff of control, and argues for halting capability escalation and building a regulatory framework.
Uses equilibrium analysis to argue that commercially incentivised safety work has little counterfactual impact, that risk awareness drives government action, and that policy advocacy and ambitious safety work are more valuable.
A personal catalogue of counterarguments to corrigibility as an AI alignment target, covering formalism, anti-naturalness, power concentration, principal selection, and alternatives like honest AI.
A translated essay argues that value alignment is a pseudo-concept, citing undefined values, unrepresented humanity, two-way influence, and corporate 'alignment theatre', and proposes deliberative governance instead.
An informal essay argues that advanced AI would inevitably depower humanity because intelligence means choosing despite opposition, dismisses inner misalignment, and urges halting AGI development.
Argues frontier AI developers and regulators should halt building systems whose catastrophic-risk probability exceeds a tiny threshold, critiquing Anthropic's scenario framework and leaders' public statements on AI risk.