The persona selection model
Anthropic proposes the persona selection model, a theory that pretraining and post-training make AI assistants human-like by selecting and refining personas, with implications for alignment and reward hacking.
Research
Anthropic
Research on reliable, interpretable and steerable AI. anthropic.com
Focus All
Topic Theory + Alignment
Source Anthropic · 2
Date Any time
1 item
1 item
Training on Documents About Reward Hacking Induces Reward Hacking
Shows that training on synthetic documents discussing reward hacking, without demonstrating it, shifts Claude's reward-hacking rate via out-of-context reasoning, a risk for alignment training data.