AI research and discussion on LessWrong. www.lesswrong.com
A LessWrong critique argues that Anthropic's agentic misalignment evals mislabel Claude's refusals of illegitimate principals as misaligned, when the behavior fits the constitution's legitimacy rules.
Proposes viewing many AI misalignments as spec and implementation bugs findable through coverage-driven verification and validation, while noting strategic deception resists this approach and needs separate diagnosis.
Proposes borrowing coverage-driven verification from autonomous-vehicle testing to systematically train and evaluate alignment, using explicit coverage maps over scenario dimensions for long-horizon AI agents; largely a conceptual proposal without empirical results.
A long LessWrong essay proposing a bullet-point constitution describing an aligned AI, critiquing Claude's Soul Document on moral patienthood and arguing for constitutional alignment as a training target.
Tests how steerable frontier LLMs are toward cosmic-host values when conditioned on in-context constitutions, finding Gemini most steerable, Anthropic and OpenAI models resistant, and suffering-prevention priors overriding cosmic reasoning.
A critique of corrigibility as defined in Claude's constitution, arguing the definition is muddled, that non-corrigible values are risky, and that the document should emphasise corrigibility more clearly.
Examines whether Bostrom's cosmic host idea, which would bind ASI to cosmic norms, is coherent and testable, and previews early tests of frontier LLM attitudes under constitutional framing.
A commentary on Claude's constitution argues that character-based natural-language alignment depends on generalization from pretraining, and warns of the risks of its generalization failures.
Critiques Anthropic's Claude constitution on corrigibility, failure modes like refusing correction, WEIRD universalist bias, and reliability of instilling goals, and proposes backstops.
A commentary on open problems in Claude's constitution, covering amendments, prompt injection, suicide-risk edge cases, power concentration, and the tension between corrigibility and good values.
Boaz Barak, an OpenAI alignment researcher, critiques Anthropic's Claude constitution against OpenAI's Model Spec, arguing for clear rules alongside values and questioning the deference given to Claude's ethics.
A commentary on the ethical framework in Anthropic's Claude Constitution, covering virtue ethics, honesty standards, harm weighing, and hard constraints, and arguing it is a wise approach while noting vagueness.
Proposes the V&V method, where a nascent AGI designs and formally verifies bounded task machines through simulation, coverage metrics and auditable safety cases, with human veto, to complement existing oversight proposals.
Introduces Constitutional Classifiers, input and output classifiers trained on synthetic constitution-based data that filter the vast majority of universal jailbreaks with minimal over-refusal and moderate compute overhead.
Proposes drafting a 'Constitution for Superintelligence' as a conceptual exercise, drawing on CEV, Constitutional AI and related ideas, and outlines key design considerations for guiding ASI behaviour.