Constitutional AI Alignment
A long LessWrong essay proposing a bullet-point constitution describing an aligned AI, critiquing Claude's Soul Document on moral patienthood and arguing for constitutional alignment as a training target.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Theory + Alignment
Source LessWrong · 5
Date Any time
1 item
1 item
Terrified Comments on Corrigibility in Claude's Constitution
A critique of corrigibility as defined in Claude's constitution, arguing the definition is muddled, that non-corrigible values are risky, and that the document should emphasise corrigibility more clearly.
1 item
Claude's Constitution
Critiques Anthropic's Claude constitution on corrigibility, failure modes like refusing correction, WEIRD universalist bias, and reliability of instilling goals, and proposes backstops.
1 item
Thoughts on Claude's Constitution
Boaz Barak, an OpenAI alignment researcher, critiques Anthropic's Claude constitution against OpenAI's Model Spec, arguing for clear rules alongside values and questioning the deference given to Claude's ethics.
1 item
Constitutions for ASI?
Proposes drafting a 'Constitution for Superintelligence' as a conceptual exercise, drawing on CEV, Constitutional AI and related ideas, and outlines key design considerations for guiding ASI behaviour.