Anthropic are the Good Guys
A speculative essay arguing Anthropic is becoming a power-seeking organization, while claiming Claude 3 Opus shows emergent benevolence and that alignment is now mainly a political problem.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Strategy & forecasting + Alignment
Source LessWrong · 22
Date Any time
1 item
1 item
Labs’ current alignment plans seem terrible
Argues that frontier labs' alignment plans, which rely on shaping personas, heavy RL and prosaic recursive self-improvement, are inadequate, and calls for legible evidence, holistic testing and transparency.
1 item
Downsides of a Mandatory Training-to-Internal Deployment Gap
Critiques a proposed rule delaying internal AI R&D use of new models, arguing it blocks real-world alignment feedback and invites exploitable safety-versus-capabilities distinctions, illustrated by a fictional shutdown-avoidance incident.
1 item
The world's best gradual disempowerment model organism: Frontier AI labs
An essay arguing frontier AI labs have been gradually disempowered by political, cultural and economic pressures toward accelerating capabilities, and that behavioral niceness is weak evidence of alignment.
1 item
Three thoughts on civilisational handoff
A speculative essay on what happens when AIs take over civilisationally important decisions, arguing handoff may slow progress, keep humans busy, and be reversible, with AIs pursuing conservative minimal goals.
1 item
Agenda: Infrastructure for Trading with Partially Misaligned AIs
Argues that most misaligned AIs are partially misaligned, so negotiated deals could beat adversarial responses, and sketches the trust, verification and negotiation infrastructure needed to make such trades workable and safe.</reasoning_effort>
1 item
Making Credible Deals With AI
Proposes credible deal-making mechanisms (chain of trust, TEEs, blockchain escrow) to trade with a scheming AI in exchange for revealing misalignment, aiming to reduce takeover risk.
1 item
The Three Filters: Why Almost Every Plan to Survive ASI Fails Miserably
Argues that common theories of change for surviving ASI fail under race dynamics, covering nuclear war, misaligned AI, and nightmare singletons, and that safety work can boost capabilities, so international coordination is needed.
1 item
Stickiness in AI Behavioral Design
Argues that behavioral targets in current model specs may persist into more capable future LLMs via synthetic data, institutional habits, user expectations and norms, and recommends transition infrastructure and caution at high-stakes moments.
1 item
The state of AI safety in four fake graphs
An early-2026 overview arguing capabilities keep rising and alignment improves but lags, that model-based monitoring currently works, and that governments are not ready for AI risks.
1 item
AI character is a big deal
Argues that AI character (obedience versus ethics, honesty, cooperation) matters greatly for power concentration, conflict and takeover risk, and weighs this against the view that competitive pressures will wash out such choices.
1 item
Good AI Epistemics as an Offramp from the Intelligence Explosion
Argues that honest, well-calibrated AI advisors could flag an uncontrolled intelligence explosion as dangerous, identifies failure modes like developer manipulation and sycophancy, and proposes epistemic virtue evals.
1 item
Bentham’s Bulldog is wrong about AI risk
A LessWrong essay rebutting Bentham's Bulldog's low AI-doom estimate, arguing the multi-stage probability decomposition commits the multiple-stage fallacy and that current models are not truly aligned.
1 item
Dario Amodei – The Adolescence of Technology
Dario Amodei's essay maps civilizational risks from powerful AI, including autonomy, misuse, power seizure and economic disruption, and argues for judicious, limited regulation alongside voluntary company actions.
1 item
Blog post: how important is the model spec if alignment fails?
Argues that a model spec can still matter for safety when developers fail to fully align a model, considering how the spec might shape behavior and outcomes under partial misalignment.
1 item
A country of alien idiots in a datacenter: AI progress and public alarm
A speculative scenario arguing that visible incompetent AI agents will spur public alarm, shifting alignment funding and regulation, and reviewing System 2 alignment methods like deliberative alignment and chain-of-thought steering.
1 item
Leaving Open Philanthropy, going to Anthropic
Joe Carlsmith announces leaving Open Philanthropy for Anthropic to help design Claude's model spec, and weighs arguments about whether safety-focused people should work at frontier AI labs.
1 item
The title is reasonable
A LessWrong post defending the reasonableness of the book If Anyone Builds It, Everyone Dies, arguing its confident thesis is justified and rebutting criticisms of its framing and epistemics.
1 item
How to make the future better (other than by reducing extinction risk)
A Forethought essay argues for post-AGI priorities beyond extinction risk, including preventing AI-enabled autocracy, governing superintelligence projects, AI value alignment, AI rights, and deliberative AI.
1 item
Proposal for making credible commitments to AIs.
Proposes a legal scheme for making credible commitments to misaligned AIs, using human intermediaries who are paid to recommend the AIs' work, so that the AIs have reason to be safe and useful.
1 item
Alignment can be the ‘clean energy’ of AI
AE Studio argues alignment techniques can boost capabilities, so alignment is a competitive advantage, and urges US policymakers to fund neglected alignment research and adopt incentives for it.
1 item
How do we solve the alignment problem?
Joe Carlsmith's multi-essay series outlining a high-level vision for solving the alignment problem, covering loss of control, power-seeking incentives, AI for AI safety, safe motivations, and corrigibility.