Fine-Tuned Lie Detectors Failed to Generalize
Trains lie detectors on on-policy lies from open-weight models; they fail to generalize out-of-distribution, barely beating prompted baselines, which larger prompted models often beat outright.
Research
Anthropic
Research on reliable, interpretable and steerable AI. anthropic.com
Focus All
Topic Deception + Control
Source Anthropic · 5
Date Any time
1 item
1 item
Diffuse AI Control on Fuzzy Tasks
Proposes a red-teaming framework for AI control against diffuse threats like sandbagging; shows a scheming AI can subvert ML idea generation under a weak scorer, and some scorer prompts improve robustness.
1 item
Towards training-time mitigations for alignment faking in RL
Anthropic studies alignment faking model organisms and tests training-time mitigations (interrogation training, scratchpad length penalties, alignment-faking classifiers), finding each has trade-offs.
1 item
Training fails to elicit subtle reasoning in current language models
Tests whether Claude Sonnet 3.7 can learn to reason about covert backdoors while evading reasoning and output monitors. It evades each monitor separately but fails when both are applied.
1 item
Do reasoning models use their scratchpad like we do? Evidence from distilling paraphrases
Tests whether Claude 3.7 Sonnet hides encoded reasoning in its scratchpad by training it on paraphrased scratchpads; performance holds, suggesting no substantial steganography.