How Hard a Problem is Alignment? (My Opinionated Answer)
An opinionated essay estimating how hard AI alignment is by comparing person-years of effort against steam engine safety, Apollo, and P vs NP, and arguing for accelerating the field's growth.
Research / Sources
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic Strategy & forecasting
Date Any time
How do we (more) safely defer to AIs?
Discusses how to safely defer to increasingly capable AIs for AI safety and strategy work, covering the basin of good deference, required alignment and capability profiles, and the lead time needed to manage exogenous risks.
Will we get automated alignment research before an AI Takeoff?
Forecasts that AI will automate capabilities research before safety research, argues this ordering is risky, and proposes differential-acceleration steps such as building safety-research automation tools and granting safety researchers early model access.
AI for AI safety
Carlsmith argues that using frontier AI labor to strengthen safety progress, risk evaluation, and capability restraint, especially automated alignment research, is crucial for avoiding loss of control from an AI capability feedback loop.