Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Proposes borrowing coverage-driven verification from autonomous-vehicle testing to systematically train and evaluate alignment, using explicit coverage maps over scenario dimensions for long-horizon AI agents; largely a conceptual proposal without empirical results.
A long LessWrong essay proposing a bullet-point constitution describing an aligned AI, critiquing Claude's Soul Document on moral patienthood and arguing for constitutional alignment as a training target.
Tests how steerable frontier LLMs are toward cosmic-host values when conditioned on in-context constitutions, finding Gemini most steerable, Anthropic and OpenAI models resistant, and suffering-prevention priors overriding cosmic reasoning.
A critique of corrigibility as defined in Claude's constitution, arguing the definition is muddled, that non-corrigible values are risky, and that the document should emphasise corrigibility more clearly.
A commentary on Claude's constitution argues that character-based natural-language alignment depends on generalization from pretraining, and warns of the risks of its generalization failures.
Critiques Anthropic's Claude constitution on corrigibility, failure modes like refusing correction, WEIRD universalist bias, and reliability of instilling goals, and proposes backstops.
A commentary on open problems in Claude's constitution, covering amendments, prompt injection, suicide-risk edge cases, power concentration, and the tension between corrigibility and good values.
Boaz Barak, an OpenAI alignment researcher, critiques Anthropic's Claude constitution against OpenAI's Model Spec, arguing for clear rules alongside values and questioning the deference given to Claude's ethics.
A commentary on the ethical framework in Anthropic's Claude Constitution, covering virtue ethics, honesty standards, harm weighing, and hard constraints, and arguing it is a wise approach while noting vagueness.
Introduces Constitutional Classifiers, input and output classifiers trained on synthetic constitution-based data that filter the vast majority of universal jailbreaks with minimal over-refusal and moderate compute overhead.
Proposes drafting a 'Constitution for Superintelligence' as a conceptual exercise, drawing on CEV, Constitutional AI and related ideas, and outlines key design considerations for guiding ASI behaviour.