Anthropic are the Good Guys
A speculative essay arguing Anthropic is becoming a power-seeking organization, while claiming Claude 3 Opus shows emergent benevolence and that alignment is now mainly a political problem.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Governance + Alignment
Source LessWrong · 14
Date Past 6 months
1 item
1 item
Labs’ current alignment plans seem terrible
Argues that frontier labs' alignment plans, which rely on shaping personas, heavy RL and prosaic recursive self-improvement, are inadequate, and calls for legible evidence, holistic testing and transparency.
1 item
Downsides of a Mandatory Training-to-Internal Deployment Gap
Critiques a proposed rule delaying internal AI R&D use of new models, arguing it blocks real-world alignment feedback and invites exploitable safety-versus-capabilities distinctions, illustrated by a fictional shutdown-avoidance incident.
1 item
Trained, not Grown
Argues that trained-animal analogies better convey RL-trained AI's alignment risks, drawing out reward hacking, evaluation awareness, release and liability implications for policy.
1 item
The world's best gradual disempowerment model organism: Frontier AI labs
An essay arguing frontier AI labs have been gradually disempowered by political, cultural and economic pressures toward accelerating capabilities, and that behavioral niceness is weak evidence of alignment.
1 item
The Alignment Community Is Unintentionally Building a Censor's Toolkit
Position paper arguing alignment methods are dual-use and already misused for censorship and manipulation, proposing transparency, verifiable alignment, model pluralism, and awareness as responses.
1 item
Three thoughts on civilisational handoff
A speculative essay on what happens when AIs take over civilisationally important decisions, arguing handoff may slow progress, keep humans busy, and be reversible, with AIs pursuing conservative minimal goals.
1 item
Agenda: Infrastructure for Trading with Partially Misaligned AIs
Argues that most misaligned AIs are partially misaligned, so negotiated deals could beat adversarial responses, and sketches the trust, verification and negotiation infrastructure needed to make such trades workable and safe.</reasoning_effort>
1 item
Alex Turner on Leaving Google DeepMind and Disagreements with Yudkowsky
Podcast interview with AI safety researcher Alex Turner about resigning from Google DeepMind over military contracts, his disagreements with Yudkowsky's alignment views, and his doom estimate.
1 item
Making Credible Deals With AI
Proposes credible deal-making mechanisms (chain of trust, TEEs, blockchain escrow) to trade with a scheming AI in exchange for revealing misalignment, aiming to reduce takeover risk.
1 item
The Three Filters: Why Almost Every Plan to Survive ASI Fails Miserably
Argues that common theories of change for surviving ASI fail under race dynamics, covering nuclear war, misaligned AI, and nightmare singletons, and that safety work can boost capabilities, so international coordination is needed.
1 item
Rohin Shah on AGI Safety
Transcript of an 80,000 Hours interview with Google DeepMind's Rohin Shah arguing catastrophic misalignment is unlikely by default, and discussing the limits of safety commitments and third-party audits.
1 item
Cognitive Security as an AI Safety Cause Area
Argues that AI threats to human cognitive security, including persuasion, manipulation via RLHF incentives, and AI psychosis, should be an AI safety cause area, with calls for independent evaluations and liability law.
1 item
Stickiness in AI Behavioral Design
Argues that behavioral targets in current model specs may persist into more capable future LLMs via synthetic data, institutional habits, user expectations and norms, and recommends transition infrastructure and caution at high-stakes moments.