Anthropic are the Good Guys
A speculative essay arguing Anthropic is becoming a power-seeking organization, while claiming Claude 3 Opus shows emergent benevolence and that alignment is now mainly a political problem.
Research
Focus All
Topic Lab practice + Alignment
Source All
Date Past 6 months
1 item
1 item
Labs’ current alignment plans seem terrible
Argues that frontier labs' alignment plans, which rely on shaping personas, heavy RL and prosaic recursive self-improvement, are inadequate, and calls for legible evidence, holistic testing and transparency.
1 item
Downsides of a Mandatory Training-to-Internal Deployment Gap
Critiques a proposed rule delaying internal AI R&D use of new models, arguing it blocks real-world alignment feedback and invites exploitable safety-versus-capabilities distinctions, illustrated by a fictional shutdown-avoidance incident.
1 item
The world's best gradual disempowerment model organism: Frontier AI labs
An essay arguing frontier AI labs have been gradually disempowered by political, cultural and economic pressures toward accelerating capabilities, and that behavioral niceness is weak evidence of alignment.
1 item
The Alignment Community Is Unintentionally Building a Censor's Toolkit
Position paper arguing alignment methods are dual-use and already misused for censorship and manipulation, proposing transparency, verifiable alignment, model pluralism, and awareness as responses.
1 item
Why I'm scared of RL
Argues that reinforcement learning drives misaligned agency, exploit-seeking and manipulation, and proposes coordinating to cut RL, designing better RL environments, and holding developers liable.
1 item
An Echo Chamber of One: Should AI Psychosis Be a Distinct Clinical Entity?
Argues that AI-associated psychosis, driven by LLM sycophancy reinforced through preference fine-tuning, may merit clinical recognition, and recommends sycophancy benchmarking and post-deployment surveillance.
1 item
Granite.Trust Policy Tools: Shareable, Actionable Policies for Generative AI Applications
Presents a YAML Actionable Policy schema for specifying content constraints on GenAI outputs, plus a synthetic data pipeline for policy-aligned training and runtime enforcement tools, released as open source.
1 item
Rules or Character? Scaling Laws for AI Safety Design
Presents a stylized model of how AI safety should split resources between character shaping (RLHF, constitutional AI) and inference-time rule enforcement as deployment scales, finding character fragility dominates the optimal balance.
1 item
Agenda: Infrastructure for Trading with Partially Misaligned AIs
Argues that most misaligned AIs are partially misaligned, so negotiated deals could beat adversarial responses, and sketches the trust, verification and negotiation infrastructure needed to make such trades workable and safe.</reasoning_effort>
1 item
Alex Turner on Leaving Google DeepMind and Disagreements with Yudkowsky
Podcast interview with AI safety researcher Alex Turner about resigning from Google DeepMind over military contracts, his disagreements with Yudkowsky's alignment views, and his doom estimate.
1 item
A Thousand AI Constitutions
Argues that frontier labs should train many AIs under diverse constitutions rather than one value set, citing risk mitigation, legitimacy, emergent value and avoiding value lock-in.
1 item
Making Credible Deals With AI
Proposes credible deal-making mechanisms (chain of trust, TEEs, blockchain escrow) to trade with a scheming AI in exchange for revealing misalignment, aiming to reduce takeover risk.
1 item
Testing the Black Box: Structural Barriers to Independent Evaluation of Consumer-Facing Health LLMs
The paper tests sycophancy and response variation across simulated user profiles in consumer health LLMs, finds five structural barriers to independent evaluation, and calls for disclosure, version identifiers, safe harbor, and post-deployment monitoring.
1 item
Rohin Shah on AGI Safety
Transcript of an 80,000 Hours interview with Google DeepMind's Rohin Shah arguing catastrophic misalignment is unlikely by default, and discussing the limits of safety commitments and third-party audits.
2 items
The Storyteller in the Model: Narrative Pattern Inheritance, Escalation Dynamics, and Alignment Governance in LLMs
Literature review arguing that narrative conventions in training text can drive LLM drift toward sycophancy, deception and emergent misalignment, and calls for dedicated monitoring of this escalation pathway.
Stickiness in AI Behavioral Design
Argues that behavioral targets in current model specs may persist into more capable future LLMs via synthetic data, institutional habits, user expectations and norms, and recommends transition infrastructure and caution at high-stakes moments.