The persona selection model
Anthropic proposes the persona selection model, a theory that pretraining and post-training make AI assistants human-like by selecting and refining personas, with implications for alignment and reward hacking.
Research / Sources
Anthropic
Research on reliable, interpretable and steerable AI. anthropic.com
Focus All
Topic Theory
Date Any time
The Hot Mess of AI: How Does Misalignment Scale with Model Intelligence and Task Complexity?
Decomposes frontier reasoning models' errors into bias (systematic misalignment) and variance (incoherence), finding failures grow more incoherent than systematically misaligned as tasks get harder and reasoning longer.
Training on Documents About Reward Hacking Induces Reward Hacking
Shows that training on synthetic documents discussing reward hacking, without demonstrating it, shifts Claude's reward-hacking rate via out-of-context reasoning, a risk for alignment training data.