Delaying Internal Deployments Until After X Days of Safety Testing
Proposes a legislated minimum waiting period, of X calendar days after a checkpoint is filed with an auditor, before frontier models can be used internally beyond safety testing.
Research / Sources
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic Regulation & law
Date Any time
Why I'm scared of RL
Argues that reinforcement learning drives misaligned agency, exploit-seeking and manipulation, and proposes coordinating to cut RL, designing better RL environments, and holding developers liable.
A Red Line and Oversight Framework for Government AI Contracts
Proposes a corporate red-line and oversight framework for government AI contracts, covering human control over targeting and untargeted profiling, with a review body that issues findings but cannot block deals.
The current bottleneck is political will, not research
Argues the main bottleneck for AI safety is policymakers' lack of awareness of catastrophic risk, so advocacy and engagement should be prioritised over research, with supporting evidence on NGO submissions and enforcement gaps.
ControlAI 2025 Impact Report: our progress toward an international ban on ASI
ControlAI's 2025 impact report describes lobbying lawmakers in the UK, US, Canada and Germany to support a ban on superintelligence development, citing extinction-risk concerns and early policy results.
AXRP Episode 48 - Guive Assadi on AI Property Rights
Podcast interview with Guive Assadi arguing that granting AIs property rights would deter them from stealing from or killing humans, since expropriation would undermine the property system they rely on.
AXRP Episode 44 - Peter Salib on AI Rights for Human Safety
Interview with law professor Peter Salib on his paper arguing that granting AIs contract, property, and tort rights could reduce incentives for AI-human conflict and takeover, via game-theoretic reasoning.
The 80/20 playbook for mitigating AI scheming in 2025
Blog post surveying a defense-in-depth toolkit for mitigating AI scheming, covering architecture choices, control, white-box detection, prompting, and enforcement, and estimating risk reduction from roughly 25% to 10%.
The best approaches for mitigating "the intelligence curse" (or gradual disempowerment); my quick guesses at the best object-level interventions
Proposes and ranks object-level interventions against gradual disempowerment from AI, mainly mandatory model interoperability for alignment and fine-tuning, aligned personal AI representatives, and societal awareness, while deprioritizing other approaches.
We Have No Plan for Preventing Loss of Control in Open Models
Argues that loss-of-control risk from powerful open-weight models is under-addressed, that labs cannot be relied on to disclose dangers, and that global open-model regulation would be very hard to pass and enforce.
Training Data Attribution: Examining Its Adoption & Use Cases
A report assessing whether training data attribution can become practical on frontier LLMs, its benefits and disclosure barriers, and its potential for reducing large-scale AI risk via interpretability.