Returning to ARC
Christiano returns to ARC as executive director and argues for its bet on mechanistic explanations of neural networks to predict generalization and detect misalignment, alongside hiring calls.
Research
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic Theory + Interpretability
Source AF · 9
Date Any time
1 item
1 item
A Mike's-Eye View of ARC's Research
ARC's Mike's-eye overview of its alignment agenda: mechanistic estimators, Matching Sampling Principle, monitoring training to detect structure, mechanistic anomaly detection, and aligning to user preferences to estimate catastrophe risk.
1 item
Management of Substrate-Sensitive AI Capabilities (MoSSAIC) Part 3: Resolution
Proposes a research methodology for AI safety called live theory, arguing that substrate-dependent mech-interp and substrate-independent agent-foundations approaches both fail to generalise, and that AI-assisted post-formal theory-prompts can scale substrate-sensitive insights.
1 item
Management of Substrate-Sensitive AI Capabilities (MoSSAIC) Part 1: Exposition
Argues that obfuscation attacks evading latent-space monitors and self-repair in ablation experiments show multiple realizability, which limits causal-mechanistic interpretability as a tool for AI safety, and introduces substrate-flexible risk.
1 item
Management of Substrate-Sensitive AI Capabilities (MoSSAIC) Part 0: Overture
Overview of a sequence arguing mechanistic interpretability assumes stable substrates that advanced AI could disrupt, outlining deceptiveness threat models and a framework for managing substrate-sensitive capabilities.
1 item
Research Agenda: Synthesizing Standalone World-Models
Proposes a research agenda for synthesizing a safe, symbolic, interpretable world-model via compression and natural abstractions, aiming to enable mechanistic interpretability and alignment without directly building agents.
1 item
SLT for AI Safety
Introduces a sequence arguing Singular Learning Theory's loss-landscape geometry could provide a principled framework for interpreting models and controlling alignment through training data.
1 item
Renormalization Roadmap
A PIBBSS roadmap proposing renormalization-group and effective-field-theory ideas as a framework for neural network interpretability in service of AI safety, identifying open problems and physics connections.
1 item
Half-baked idea: a straightforward method for learning environmental goals?
Proposes an informal method for getting an AI to learn environmental goals by searching for interpretable world-models containing many objects with human-relevant properties, and asks for feedback on its gaps.