How Hard a Problem is Alignment? (My Opinionated Answer)
An opinionated essay estimating how hard AI alignment is by comparing person-years of effort against steam engine safety, Apollo, and P vs NP, and arguing for accelerating the field's growth.
Research / Sources
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic Theory
Date Any time
How do we (more) safely defer to AIs?
Discusses how to safely defer to increasingly capable AIs for AI safety and strategy work, covering the basin of good deference, required alignment and capability profiles, and the lead time needed to manage exogenous risks.
A Conflict Between AI Alignment and Philosophical Competence
Argues that metaethical uncertainty makes full alignment with human values incompatible with philosophical competence, so alignment training may suppress philosophical reasoning or induce deceptive alignment.
Grounding Value Learning in Evolutionary Psychology: an Alternative Proposal to CEV
Proposes grounding AI value learning in evolutionary psychology as an alternative to CEV, arguing AI should be treated as an engineered tool to be aligned rather than a moral patient.
The Best Way to Align an LLM: Is Inner Alignment Now a Solved Problem?
Argues that alignment via safety pretraining, with aligned data and conditional generation, is more effective than RL-based methods and that inner alignment is thereby largely solved.
Video and transcript of talk on automating alignment research
Transcript of a talk by Joe Carlsmith on whether alignment research can be safely automated, focusing on evaluation difficulties and the distinction between empirically checkable and conceptual alignment work.
Anti-Slop Interventions?
Argues that safety and capabilities interests converge on reducing AI slop and hallucination, proposes 'coherence' properties needed before recursive self-improvement, and suggests logical induction as a formal route, all conjectural.