Highlights from Garrison Lovely's "Obsolete"
A review of Garrison Lovely's book on AI's labor-displacement risks, arguing for pause advocacy, liability for uninsurable risks, and stronger governance against lobbying and unchecked deployment.
Research / Sources
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Strategy & forecasting
Date Any time
D̶o̶n̶’̶t̵⠀Can’t Look Up
Argues that AI development is hard to pause because of a visibility problem (unclear risks) and a structural problem (state incentives in anarchic international order), and proposes what coordination would need.
1000 Years of AI Safety
Models how much AI safety research a multi-year pause among top labs could produce, including verification bottlenecks and Plan A scaling tradeoffs, using a semi-endogenous growth framework.
"Holy shit" graphs and how to get more of them
Argues that labs should publish auditable capability and deployment metrics, such as internal spending, revenue and AI R&D automation trends, so policymakers can track frontier risks like loss of control and gradual disempowerment.
Pacing the Frontier: A Framework & Research Agenda
Proposes pacing AI development, meaning interventions that moderate its pace, as a dedicated research field, covering incentives, regulatory and technical tools, and the full lifecycle of interventions.
Let's talk about the AI coordination problem
Argues that coordination among AI developers to avoid building dangerous AI deserves serious, concrete discussion like a real geopolitical negotiation, and that leaders should explain why they consider it infeasible.
On using crises to shift political will for AI
Argues that AI safety incidents and their felt urgency can shift political will for frontier AI regulation, drawing on precedents like nuclear treaties, Montreal Protocol, and COVID responses, and urges advocates to exploit them.
AI-amplified democratic backsliding: an exploration
Builds a pilot index scoring 27 countries' structural vulnerability to AI-amplified democratic backsliding across five drivers, finding liberal democracies most resilient and polarization a key weakness.
Don't normalize a permanent underclass (even a rich one)
Argues that post-AGI cosmic resources should not be allocated by pre-AGI wealth, since tradable claims would create a permanent underclass and race incentives, and proposes inalienability and staged allocation as alternatives.
AI Safety Policy Needs to train Legal Practitioners
An essay arguing that AI safety policy needs legal practitioners who can implement regulation, using GDPR's Data Protection Officer failures as a warning and describing efforts to train such people in AI safety.
Model access for third-parties — it's a big deal!
Argues external safety researchers and auditors need model access comparable to frontier-lab insiders, analysing access gaps from non-release, deployment lag, safeguards, costs and elicitation.
The Invisible Side of AI Governance
A LessWrong strategy post arguing that much impactful AI governance work is invisible insider policy-making in executive branches and multilateral forums, and that the safety community underinvests in it.
The Three Filters: Why Almost Every Plan to Survive ASI Fails Miserably
Argues that common theories of change for surviving ASI fail under race dynamics, covering nuclear war, misaligned AI, and nightmare singletons, and that safety work can boost capabilities, so international coordination is needed.
How valuable are weak AI safety regulations?
Argues that weak AI regulations such as GPU export controls, mandatory safety testing and whistleblower protections can modestly reduce extinction risk, weighing their routes to impact against downsides.
What if Anthropic unilaterally paused capabilities development right now?
A LessWrong opinion piece arguing that Anthropic should unilaterally pause frontier capabilities development for safety, analysing how this would affect other labs, governments, and Anthropic's IPO.
Engineering a Safer World: Risk Modelling — and Safety Engineering? — for AI Loss of Control
Applies Leveson's systems-safety thinking to AI loss of control, modelling how breaches at each oversight level can escalate into vicious cycles and how AI could also strengthen governance.
The case for fine-grained tracking of compute for AI
Argues that FLOP/s and bandwidth proxies no longer track AI compute well and proposes architecture-specific compute scaling models, to improve forecasts and compute governance thresholds.
Voters are surprisingly open to talking about AI risk
A LessWrong post arguing that US voters are unexpectedly open to discussing AI existential risk, and offering canvassing tactics for AI-safety-friendly candidates like Alex Bores.
Could Frontier AI Researchers Collectively Slow the Race? A Conditional Pledge Mechanism
Proposes a conditional pledge mechanism in which frontier AI researchers would collectively step away from capability work if enough peers across labs sign, and seeks feedback on feasibility and barriers.
Dawn of the "national security" tier of AI
Speculative commentary arguing that a US government 'national security' tier of frontier AI, with pre-release vetting and public-private partnerships, is emerging after Claude Mythos's hacking capabilities, and naming likely political players.
Only Politics Can Prevent Extinction*
An opinion post arguing that electoral politics and a pro-AI-safety political movement are necessary to get strict AI regulation enacted, citing historical cases of popular policies stalled by legislative inertia.
Alignment to Evil
Argues that an ASI developer's commitment to the common good matters for utopia, and that authoritarian capture of such organisations could make even a shutdown-based off-ramp lead to dystopia.
AI Researchers and Executives Continue to Underestimate the Near-Future Risks of Open Models
Argues that open-weight models undermine the defenses proposed for near-future AI risks, since guardrails can be fine-tuned away, use cannot be monitored, and bad actors have access by default.
Building Technology to Drive AI Governance
Argues that technical builders should create measurement, cheap evaluation, and privacy-preserving audit tools to drive AI governance, drawing lessons from climate, food safety, and pandemic response.
Long-term risks from ideological fanaticism
Argues ideological fanaticism is a major driver of atrocities and existential risk, especially via misuse of advanced AI, and proposes compute governance and anti-fanatical interventions.
What We Learned from Briefing 140+ Lawmakers on the Threat from AI
ControlAI's director reflects on briefing over 140 UK parliamentarians about superintelligence extinction risk, describing how cross-party support grew and lessons on advocacy and policy design for a prohibition.
The third option in alignment
Speculative essay arguing that humans might voluntarily hand control to AI, possibly via insiders training deliberately misaligned models, and that GPU embargoes and alignment research would not stop this.
New Report: An International Agreement to Prevent the Premature Creation of Artificial Superintelligence
MIRI's Technical Governance Team proposes an international agreement to halt frontier AI advancement toward superintelligence via compute limits, chip tracking, and research restrictions, justified by misalignment and catastrophic risk.
Hardening against AI takeover is difficult, but we should try
Commentary on a RAND report claiming a superintelligence could be contained by hardening world infrastructure against harm, arguing the proposed safeguards are weak and implementation and buy-in remain major obstacles.
Falling AI Costs and the Proliferation of Offensive Capabilities
Essay arguing that falling AI development costs and easy model theft will lead to proliferation of offensive capabilities, drawing analogies to cryptography and estimating training cost trends for dangerous AI.
How an AI company CEO could quietly take over the world
A concrete scenario in which an AI company CEO uses backdoored, secretly loyal AIs to take control, with policy recommendations on monitoring, access controls and transparency at AGI labs.
Alignment Faking Demo for Congressional Staffers
A script and notes from a demo presenting alignment faking in a medical chatbot to congressional staffers, with reflections on AI safety communication and evaluation realism.
"Shut It Down" is simpler than "Controlled Takeoff"
Compares feasibility and strategy of Global Shutdown versus Global Controlled Takeoff for AI, arguing controlled takeoff is not clearly more tractable and that removing economic pressure to rush is key.
Metacrisis as a Framework for AI Governance
Argues that Daniel Schmachtenberger's metacrisis theory offers a coherent framework for AI governance, identifying shared risk claims and proposing new research priorities on risk interactions, systemic change and collective intelligence.
The Problem with Defining an "AGI Ban" by Outcome (a lawyer's take).
A lawyer argues that an AGI ban must define precise, ex ante precursor capability thresholds with strict liability, rather than outcome-based definitions, drawing on nuclear treaties and EU AI Act precedents.
Good government
A speculative essay sketching values for good post-AGI government, including kindness, peace, and freedom, and arguing for more positive visions alongside risk prevention.
We should think about the pivotal act again. Here's a better version of it.
Argues that the traditional Yudkowsky-style pivotal act requires suppressing humanity and proposes a non-coercive alternative: an aligned ASI demonstrates its power, then hands control back and promotes voluntary AI regulation.
Good Ideas Aren't Enough in AI Policy
An essay arguing that AI safety advocates need political relationship-building, coalition work and pragmatic lobbying skills, not just good arguments, using the 2025 AI moratorium fight as a case study.
AI Offense Defense Balance in a Multipolar World
Analyses whether intent-aligned defensive AI can counter takeover-level adversarial AI, proposing threat models, a three-pillar defense framework, and legal constraints, and concludes the offense-defense balance remains uncertain.
Lessons from the Iraq War for AI policy
Draws lessons from the 2003 Iraq War, arguing that a catastrophic AI event could open a policy window for extreme, poorly executed responses driven by elite factions rather than public judgement.