Ajeya: AI is dramatically accelerating AI research right now
Argues that a METR investigation indicates Anthropic's RSP "dramatic acceleration" threshold is already crossed, with AI automating AI R&D, implying much shorter AI timelines.
Research
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic Lab practice + Evals
Source AF · 5
Date Any time
1 item
1 item
AI companies have started saying safeguards are load-bearing
Critiques frontier labs' safety claims on bio misuse safeguards, model-weight security, and misalignment planning, finding most claims unsupported and current plans inadequate for future risks.
2 items
When is it important that open-weight models aren't released? My thoughts on the benefits and dangers of open-weight models in response to developments in CBRN capabilities.
Argues open-weight models at the level of significantly aiding amateur bioweapon creation would cause large expected fatalities, but that their benefits for mitigating loss-of-control risk make advocacy against them unwise; recommends mitigations and honest CBRN evaluations.
AI companies' eval reports mostly don't support their claims
Critiques frontier labs' dangerous-capability eval reports, arguing they fail to justify safety claims and that poor elicitation underestimates model capabilities, especially for bio and cyber risks.
1 item
Planning for Extreme AI Risks
A strategic playbook for a hypothetical frontier developer on minimizing extreme AI takeover risks, covering goals, outcomes, and prioritization heuristics for scaling, safety preparation, and governance.