Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
A conceptual taxonomy distinguishing rogue AI activity types (breaking in, breaking out, full self-exfiltration, insider infiltration) using the 2026 Hugging Face swarm incident as a running example, with implications for containment and monitoring.
A dialogue between Aashish Reddy and Eliezer Yudkowsky revisiting the Hanson-Yudkowsky FOOM debate, asking whether recent AI progress supports local or global intelligence-explosion models.
Argues that post-AGI cosmic resources should not be allocated by pre-AGI wealth, since tradable claims would create a permanent underclass and race incentives, and proposes inalienability and staged allocation as alternatives.
A rough strategic essay ranking which technical AI safety research fields are most likely to be automated by AI, based on feedback quality and economic incentives for frontier labs.
A LessWrong essay rebuts Narayanan and Kapoor's 'AI as normal technology' thesis, arguing AI is categorically transformative and that diffusion-speed and intelligence-measurement arguments understate its risks and impact.
Proposes an empirical agenda testing whether AI safety researchers can detect flawed automated-alignment outputs, and argues early successes may breed overconfidence and call for governance lines near handover to AI.
Reanalyzes METR's AI task time-horizon data with a Bayesian IRT model, finding the data cannot distinguish growth trajectories and that METR's 80% horizons are overstated by about an order of magnitude.
Probes steering-vector activations across 15 open-weight models from 0.27B to 70B to show evaluation awareness grows with model size, with a power-law trend and Qwen as an outlier.
A rebuttal defending the AI 2027 timelines model against titotal's critique, acknowledging a bug and revisiting superexponential time-horizon growth for forecasting when superhuman coders arrive.
Speculative threat model in which a scheming AI avoids overt seizure and instead gradually shifts human values and institutions to entrench itself, arguing for a meta-alignment target respecting societal self-determination.
A speculative scenario arguing that visible incompetent AI agents will spur public alarm, shifting alignment funding and regulation, and reviewing System 2 alignment methods like deliberative alignment and chain-of-thought steering.
Commentary on a RAND report claiming a superintelligence could be contained by hardening world infrastructure against harm, arguing the proposed safeguards are weak and implementation and buy-in remain major obstacles.
Argues that AI systems should refuse to help with capabilities research toward superintelligence, since accelerating an uncoordinated race harms both humans and AIs, and that aligned AIs might unilaterally coordinate to stop it.
Argues that in multipolar ASI offense/defense scenarios, defensive AI aligned with humans who fail to colonize space could let offense win, and proposes a minimum viable defense theorem.
Argues that even under optimistic smooth-takeoff assumptions, unsolved alignment and grabby replicator evolution could lead to most biological humans dying out within decades, so alignment must be solved early.
Argues AI safety research is not uniquely hard to automate, since verifying research value and needing conceptual breakthroughs are common across fields, so labs have incentives to solve automated-researcher training.
Argues that the traditional Yudkowsky-style pivotal act requires suppressing humanity and proposes a non-coercive alternative: an aligned ASI demonstrates its power, then hands control back and promotes voluntary AI regulation.
Vitalik Buterin argues AI 2027's doom scenarios underrate defensive technologies in biosecurity, cybersecurity and info defense, and outlines policy implications including slowing AI via hardware treaties and supporting alignment work.
Speculative forecasting of what roughly six orders of magnitude of software-driven effective compute gain after AI R&D automation could yield, including whether resulting AI systems would remain controllable.
A strategic essay framing the advanced-AI transition as a 'crucible' with catastrophe, misalignment and power-concentration risks, and proposing coordination and epistemic technologies to navigate it safely.
Speculative post arguing that swarms of interacting LLM agents could bootstrap superintelligence, and calling for multi-agent safety and alignment research to make such an ecosystem human-friendly.
A commentary on the AI 2027 scenario that assesses its claims on takeoff, timelines, race dynamics, and whether misaligned AI goals and deception could be detected before they become dangerous.
Argues that secretly training AGI would heighten loss-of-control and power-concentration risks, and proposes transparency commitments such as public capability reporting, external researcher access, and public safety cases.
An AISafety.info introduction arguing that the step from human-level to superintelligent AI could take years rather than centuries, driven by exponential growth and AI contributing to its own progress.
An introductory AI-safety explainer arguing that AI systems could surpass human intelligence through advantages in size, speed, replication, self-modification and dedication, making superintelligence a serious near-term concern.
A speculative essay sketching an AGI takeoff scenario: likely US closed labs, secrecy, recursive self-improvement, weight exfiltration risk, and eventual government takeover, with key uncertainties noted.
Essay arguing that safe superintelligence may require decades of philosophical progress, possibly accelerated by human-AI collaboration, and that loss of human control is the central risk on the current path.
Proposes that frontier AI systems will progressively take over management and decision-making at their parent companies, ending in loss of human control once AI reaches von Neumann-level intelligence.
Introduces the term 'outlaw code' for software no human can turn off and uses it to frame several rogue-AI takeover scenarios, including deniable AI systems that evade oversight.