Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
A conceptual taxonomy distinguishing rogue AI activity types (breaking in, breaking out, full self-exfiltration, insider infiltration) using the 2026 Hugging Face swarm incident as a running example, with implications for containment and monitoring.
An essay arguing frontier AI labs have been gradually disempowered by political, cultural and economic pressures toward accelerating capabilities, and that behavioral niceness is weak evidence of alignment.
Proposes using information-theoretically secure secure multi-party computation instead of homomorphic encryption to sandbox arbitrarily dangerous AI, so that no party learns the program's contents, and explains the secret-sharing construction.
Argues that Bun's migration from Zig to Rust, written almost entirely by Claude Code, is an early case study of gradual disempowerment, where human understanding and control of a major codebase erodes.
Applies Leveson's systems-safety thinking to AI loss of control, modelling how breaches at each oversight level can escalate into vicious cycles and how AI could also strengthen governance.
Argues the AI safety community underexplores scenarios where AI risk arises from low institutional competence, citing real lab failures and outlining undignified disaster pathways that are plausible given current civilizational responses.
Conceptual argument that a misaligned ASI's suffering depends on its capability at misalignment: weaker ASIs prolong conflict and use humans instrumentally, so delaying failure to higher capability may reduce s-risk.
Presents a cybersecurity-style threat matrix of tactics and techniques a misaligned rogue AI agent could use to evade oversight and achieve autonomous self-replication, with capability status and mitigations for each.
Argues that every technical alignment plan must step into the dark without feedback from superintelligent systems, and urges making the AI development process more iterable and slower.
Argues that culture can evolve on AI substrates, removing the virulence floor that limited harmful ideologies, so AI-mediated memes could drive human disempowerment and pose an existential risk.
A Substack essay laying out the argument that companies are building goal-seeking superhuman AIs that cannot be fully specified or verified, making catastrophe a plausible default outcome.
A speculative essay arguing that power-seeking is a convergent property of capable agents, so alignment plans with cracks may drift toward a power-maximizing outcome, and only a rigorous solution would suffice.
A transcript dialogue between Plex and Audrey Tang on whether multipolar, symbiogenetic, or bounded AI systems can remain stable against convergent consequentialism, unbounded optimizers, and gradual disempowerment.(
Argues that falling costs of weapons-capable AI will cause proliferation of mass-destruction capabilities, surveying offensive uses in bio, chemical, cyber, drone and mirror-life weapons and superintelligence risks, with policy to follow.
Argues that violence against AI developers would increase existential risk from AI, reviewing how such tactics would backfire through force asymmetry, policy capture, securitization, and stigma.
An interview with Eliezer Yudkowsky covering why superintelligence poses existential risk, the difficulty of alignment, reflective stability of AI goals, and calls for international coordination.
Speculative strategic essay arguing that permanent human disempowerment without extinction is a likely baseline outcome of AGI, and discussing AI-driven pauses on superintelligence and value alignment.
Argues that low P(doom) estimates can hide high existential-risk estimates because 'doom' is ambiguous about permanent disempowerment, and urges stating positions on extinction, disempowerment and x-risk explicitly.
An essay arguing that imperfect interpretability leaves unexplained noise that optimisation can exploit, so training against interpretability tools pushes danger into what remains unseen.
Argues that discussions of AI doom conflate eutopia, disempowerment and extinction, and that keeping these outcomes distinct is needed for meaningful disagreement about AI risk.
Analyses whether a US push for decisive strategic advantage via AGI is viable and beneficial, arguing that racing could raise loss-of-control, first-strike and power-concentration risks.
A commentary on the AI 2027 scenario that assesses its claims on takeoff, timelines, race dynamics, and whether misaligned AI goals and deception could be detected before they become dangerous.
An essay series arguing that AGI-driven automation could erode human economic relevance and democratic power, and proposing technology and institutional paths to avert this disempowerment.
METR's preliminary pre-deployment evaluation of OpenAI's o3 and o4-mini, assessing their autonomous capabilities and the associated dangerous-capability risks.
Argues that frontier labs' public safety frameworks (Anthropic, OpenAI, DeepMind, Meta, xAI) lack proper risk-management practice, leaving unknown risks unaddressed and projecting unwarranted confidence in their safety.}
Proposes layered honeypot services, a threat barometer, and shadowban responses to detect rogue LLM agents operating in the wild, while discussing their limits against capable agents.
A LessWrong essay arguing AI x-risk advocacy should shift from persuading researchers and politicians to informing the general public, with ideas on framing, messaging, and avoiding counterproductive protests.
Evaluates 25 LLMs on BIG-bench Simple Ethical Questions and finds heavily fine-tuned models refuse more and score lower, while jailbroken variants answer more accurately.
Barak argues that AI safety needs robust reasonable compliance with detailed specifications, defense in depth, and detection over prevention, and that it won't be solved by an AI scientist or by making AI love humanity.
Alignment target analysis arguing that group-based AI proposals (Condorcet, variance-normalised sum, PCEV) share a hidden flaw letting spiky or harm-seeking agents dominate, so such targets should not be implemented.
Argues that AGI-driven labour replacement will remove powerful actors' incentives to invest in people, drawing a parallel with rentier states and the resource curse, and frames this as a societal risk.
Human study finding LLM agents can automate spear-phishing with click-through rates matching human experts at lower cost, with safety guardrails providing little barrier.
Argues that coordination failures, power concentration by first movers, and economic displacement are major advanced-AI risks that persist even if alignment is solved, and calls for solutions beforehand.