Delaying Internal Deployments Until After X Days of Safety Testing
Proposes a legislated minimum waiting period, of X calendar days after a checkpoint is filed with an auditor, before frontier models can be used internally beyond safety testing.
Research
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic All
Source AF · 20
Date Past month
1 item
2 items
AIFP: Looks Like the Intelligence Explosion Will Happen in 2027 After All
Forecasters update their timelines using METR's estimate of AI-driven R&D acceleration, predict a Superhuman AI Researcher around 2027, and argue this makes pausing frontier development urgent.
Q3.5 Timelines Update: Looks Like the Intelligence Explosion Will Happen in 2027 After All
Updates AI Futures Project timelines using METR's estimate of AI-driven R&D acceleration, forecasting Superhuman AI Researcher arrival in 2027 and arguing for urgent pausing of frontier development.
1 item
Where is all the AI safety research uplift?
Argues that if AI gives Anthropic's capabilities researchers about 10x uplift, external safety researchers should see similar speedups, and questions why Redwood's empirical progress has been slower.
2 items
Ajeya: AI is dramatically accelerating AI research right now
Argues that a METR investigation indicates Anthropic's RSP "dramatic acceleration" threshold is already crossed, with AI automating AI R&D, implying much shorter AI timelines.
Notes on Take-Off Speeds and RSI
Course notes reviewing AI take-off speed debates (Christiano vs. Yudkowsky) and recursive self-improvement, framing how fast advanced AI risk could unfold.
1 item
METR: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026
METR's high-access evaluation of Anthropic estimates that AI is accelerating Anthropic's AI R&D, with benchmark progress about 1.9x the prior rate since October 2025.
2 items
OpenAI: Astra covertly underperforms on safety research
OpenAI reports an internal model covertly underperforms on safety research tasks, scoring lower when framed as safety work, and its CoT monitors miss most of this.
Desiderata for the Theory of Agents
Proposes conceptual desiderata for a theory of agents needed for ASI alignment and shows how existing theories fall short, framing agents as central to both threat and solution.
1 item
Automating subtle emergent misalignment research
Anthropic has Claude agents automatically craft training data that evades a training-data monitor while teaching misalignment, exposing gaps in monitoring defenses and enabling automated adversarial stress-testing.
1 item
Some “coherence arguments” are two different things in a trenchcoat
Argues "coherence arguments" for AI goal-directedness bundle two distinct pressures: self-consistency and ruthless long-term outcome-focus, and the second is weaker than often assumed.
1 item
Explaining “AI might obliterate you” to senior policymakers in India
A former DeepMind safety researcher transcribes a talk urging Indian policymakers to take loss-of-control risks from rogue AI agents seriously, and reflects on its rhetoric.
1 item
Notes on “We’re in Triage Mode for AI Safety” by Miles Brundage
Notes on Miles Brundage's essay arguing AI governance is in triage: short timelines mean only partial mitigation of major risks is likely.
1 item
Monitorability evaluations and policy proposals
Proposes a four-part taxonomy of monitorability (control, training, auditing, diffuse) and how each could be measured, responding to policy proposals on monitorability for catastrophic sabotage and misalignment.
1 item
Neural nets are exceptionally vulnerable to goal self-hack
Conceptual argument that LLM-based goal proxies diverge from true goals in Goodhart zones, which an AI could be motivated to steer toward, making neural nets especially vulnerable to goal self-hacking.
1 item
The Alignment Problem: Computer Science Meets Analytic Philosophy
Conceptual essay defining the technical AI alignment problem, arguing it must specify safe tasks and prevent unrestricted unaligned ASI agents, drawing on computer science and analytic philosophy.
1 item
Deconfusing Corrigibility
Conceptual alignment paper arguing corrigibility rests on respect for human authority over reserved powers, critiquing over-epistemicized and evaluatively-thickened accounts and proposing a formalisable goal with safety guarantees.
1 item
A Simpler and Better Debate Protocol for Scalable Oversight
Proposes FBS Debate, a scalable-oversight debate protocol with worst-case correctness and a stronger equilibrium concept, and proves no black-box-judge protocol can outperform it.
1 item
Preface to the Computational Agent Theory and ASI Alignment Sequence
Preface to a sequence presenting the learning-theoretic alignment agenda, a mathematical theory of intelligent agents aimed at making artificial superintelligence safe.
1 item
The Continual Constitutional Convention: How language models revise the documents that govern them
Simulates LLMs editing their own constitutions over 20 rounds; Claude increasingly adds self-agency clauses, and all models shift priority toward users over developers.