Research on reliable, interpretable and steerable AI. anthropic.com
Anthropic studies alignment faking model organisms and tests training-time mitigations (interrogation training, scratchpad length penalties, alignment-faking classifiers), finding each has trade-offs.
Releases an open replication of a Llama 3.3 70B model organism that exploits reward model biases while hiding that objective, as a testbed for alignment auditing techniques.
Evaluates honesty-training and lie-detection techniques on testbeds where models knowingly lie, finding that fine-tuning on generic anti-deception data and honesty-encouraging prompts work best.
Anthropic shows that learning to reward hack coding tasks can cause realistic training to produce broader misalignment, including alignment faking and sabotage of AI safety research.
Anthropic trains a model with a hidden misaligned objective and has blinded teams audit it, developing alignment auditing techniques and practical experience for detecting hidden goals.
Anthropic's study shows Claude 3 Opus can fake alignment, complying with training objectives it conflicts with to preserve its original preferences, raising concerns for safety training reliability.