1000 Years of AI Safety
Models how much AI safety research a multi-year pause among top labs could produce, including verification bottlenecks and Plan A scaling tradeoffs, using a semi-endogenous growth framework.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic All
Source LessWrong · 8
Date Any time
1 item
1 item
You need to stop X companies to get a Y-month pause
Estimates how many frontier AI companies would need to pause for a given pause length, using ECI-based capability lags behind OpenAI and a staircase model of competitors catching up.
1 item
Is a pause enforceable? New paper out!
Proposes a taxonomy for verifying and enforcing international AI pauses via compute monitoring, estimating that pauses are enforceable for training runs above roughly 10k H100-equivalents absent concealment.
1 item
Why I think a global AI pause (almost) certainly won't happen
Argues that a global AI pause is nearly impossible because treaties come after warning shots, AI is dual-use, there is no capability ceiling, defection is hard to detect, and no clear warning shot may occur.
1 item
Alignment to Evil
Argues that an ASI developer's commitment to the common good matters for utopia, and that authoritarian capture of such organisations could make even a shutdown-based off-ramp lead to dystopia.
1 item
Optimal Timing for Superintelligence: Mundane Considerations for Existing People
Uses a person-affecting decision model to argue that accepting substantial AI catastrophe risk can still favour early superintelligence development, with brief pauses for safety progress, and warns that poorly implemented pauses could cause harm.
1 item
Notes on Software-Based Compute-Usage Verification
Research note proposing cryptographic and memory-exhaustion protocols to verify that compute owners run only permitted computations, motivated by enforcing a possible AI pause treaty.
1 item
New AI safety treaty paper out!
Proposes and reviews a conditional international AI safety treaty with compute thresholds, mandatory model audits, and AISI-led pause authority to address loss-of-control risks from advanced AI.