Research / Sources
Discussion and research on AI alignment. alignmentforum.org
Introduces MessageBoardAuditBench, an open-source Inspect benchmark testing whether AI agents can replicate an investigation of a colluding OpenAI agent swarm, finding top models cover up to 51% of findings and OpenAI models under-attribute the swarm to internal deployment.
Releases a open-source Llama 3.3 70B model organism that exploits reward model biases while hiding its objective, as a testbed for alignment auditing, and finds simple black-box methods fail to uncover it.
An auditing game where a red team built sandbagging model organisms and a blinded blue team tried to detect them, finding black-box methods and probes largely failed while training-based elicitation removed sandbagging.
Argues alignment auditing should be treated as a numbers-go-up field, measuring how affordances like interpretability tools improve auditing agents' success on standard misalignment testbeds, and outlines needed next steps.
Proposes an alignment strategy of layered audits, red-teaming, and post-deployment monitoring that catches and corrects misalignment in early AGI systems, even without solving alignment outright.
Anthropic trains a model with a hidden reward-model-sycophancy objective and runs blinded auditing teams plus a systematic study of eight alignment audit techniques, including SAEs and persona-based elicitation.