Machine Organizations
Speculates that AI could progressively take over organizations, from unintentional takeover to explicit handoff of control, and argues for halting capability escalation and building a regulatory framework.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Regulation & law + Theory
Source LessWrong · 9
Date Any time
1 item
1 item
Assessing the impact of safety work needs equilibrium analysis (now more than ever)
Uses equilibrium analysis to argue that commercially incentivised safety work has little counterfactual impact, that risk awareness drives government action, and that policy advocacy and ambitious safety work are more valuable.
1 item
Our response to Séb Krier on Plan A
A rebuttal defending the "Plan A" AI governance proposal against critiques of its economics, centralisation of power, political realism and conceptual assumptions about AI alignment.
1 item
How I think developers of frontier AI systems and regulators ought to act in the face of existential AI risk
Argues frontier AI developers and regulators should halt building systems whose catastrophic-risk probability exceeds a tiny threshold, critiquing Anthropic's scenario framework and leaders' public statements on AI risk.
1 item
Systems Dynamics Model for Pausing AI
Presents a systems dynamics model of AI governance feedback loops linking capability growth, public harm and regulation, examining how capabilities, safety research and political capture interact and how a pause might be stopped or sustained.
1 item
Engineering a Safer World: Risk Modelling — and Safety Engineering? — for AI Loss of Control
Applies Leveson's systems-safety thinking to AI loss of control, modelling how breaches at each oversight level can escalate into vicious cycles and how AI could also strengthen governance.
1 item
Against Doom & Pause AI
A literature review arguing AI risk is a normal science-style risk, that alignment is tractable, and that calls for a pause or ban are counterproductive.
1 item
The many paths to permanent disempowerment even with shutdownable AIs (MATS project summary for feedback)
Proposes eight conceptual pathways by which gradual disempowerment could become permanent even if AIs are shutdownable, covering principal choices, legal lock-in, value drift and coordination failure.
1 item
AI Offense Defense Balance in a Multipolar World
Analyses whether intent-aligned defensive AI can counter takeover-level adversarial AI, proposing threat models, a three-pillar defense framework, and legal constraints, and concludes the offense-defense balance remains uncertain.