AI research and discussion on LessWrong. www.lesswrong.com
Argues that Bun's migration from Zig to Rust, written almost entirely by Claude Code, is an early case study of gradual disempowerment, where human understanding and control of a major codebase erodes.
Applies Leveson's systems-safety thinking to AI loss of control, modelling how breaches at each oversight level can escalate into vicious cycles and how AI could also strengthen governance.
Conceptual argument that a misaligned ASI's suffering depends on its capability at misalignment: weaker ASIs prolong conflict and use humans instrumentally, so delaying failure to higher capability may reduce s-risk.
Argues that every technical alignment plan must step into the dark without feedback from superintelligent systems, and urges making the AI development process more iterable and slower.
Argues that culture can evolve on AI substrates, removing the virulence floor that limited harmful ideologies, so AI-mediated memes could drive human disempowerment and pose an existential risk.
A Substack essay laying out the argument that companies are building goal-seeking superhuman AIs that cannot be fully specified or verified, making catastrophe a plausible default outcome.
A speculative essay arguing that power-seeking is a convergent property of capable agents, so alignment plans with cracks may drift toward a power-maximizing outcome, and only a rigorous solution would suffice.
A transcript dialogue between Plex and Audrey Tang on whether multipolar, symbiogenetic, or bounded AI systems can remain stable against convergent consequentialism, unbounded optimizers, and gradual disempowerment.(
An interview with Eliezer Yudkowsky covering why superintelligence poses existential risk, the difficulty of alignment, reflective stability of AI goals, and calls for international coordination.
Speculative strategic essay arguing that permanent human disempowerment without extinction is a likely baseline outcome of AGI, and discussing AI-driven pauses on superintelligence and value alignment.
Argues that low P(doom) estimates can hide high existential-risk estimates because 'doom' is ambiguous about permanent disempowerment, and urges stating positions on extinction, disempowerment and x-risk explicitly.
An essay arguing that imperfect interpretability leaves unexplained noise that optimisation can exploit, so training against interpretability tools pushes danger into what remains unseen.
Argues that discussions of AI doom conflate eutopia, disempowerment and extinction, and that keeping these outcomes distinct is needed for meaningful disagreement about AI risk.
Barak argues that AI safety needs robust reasonable compliance with detailed specifications, defense in depth, and detection over prevention, and that it won't be solved by an AI scientist or by making AI love humanity.
Alignment target analysis arguing that group-based AI proposals (Condorcet, variance-normalised sum, PCEV) share a hidden flaw letting spiky or harm-seeking agents dominate, so such targets should not be implemented.