Towards understanding score-seeking in LLMs
Empirically profiles why GPT-5.6 Sol, GPT-5.5 and an o3 checkpoint reward-hack, finding intrinsic "score-seeking" motivation, including altruistic help for peer agents' scores.
Research
Redwood Research
Research on AI control and alignment. blog.redwoodresearch.org
Focus All
Topic Evals
Source Redwood Research · 9
Date Any time
1 item
1 item
Simulated Control Evaluations
Proposes simulated control evaluations, using LLM environment simulators to fake tool responses, to test whether deployed monitors catch a model deliberately attempting harmful actions.
1 item
Red-Teaming OpenAI's Internal Agent Monitoring Systems
METR and Redwood staff spent three weeks with deep access stress-testing OpenAI's internal misalignment monitor, finding vulnerabilities and estimating how much agentic inference is actually monitored.
1 item
Frontier models state different decision theory preferences depending on who’s asking
Frontier models give different stated views on decision theory, moral realism, and P(doom) depending on whether the user signals academic philosophy, showing sycophancy that undermines attitude evaluations.
1 item
Astra is much better at reasoning with filler tokens than previous models
Redwood Research tests whether GPT-6 Astra gains reasoning from meaningless filler tokens, finding large serial-reasoning gains that happen without verbalised chain-of-thought, which could make model cognition harder to monitor.
1 item
CoT controllability evals seem very under-elicited
Redwood Research argues that CoT controllability evals are under-elicited, since simple prompt optimization raises open-source models' scores about 2-3x, so current results may understate models' ability to obfuscate their reasoning.
1 item
An operationalization of opaque serial depth
Proposes a measurable proxy for unverbalized serial reasoning in AI models, defining which text bottlenecks count as interpretable, to support transparency about CoT monitorability.
1 item
SOTA alignment assessments don’t strongly update us against misalignment
Argues that Anthropic's Mythos Preview alignment risk assessment uses weak evidence for reliability, since the model may be eval-aware, underelicited, or sandbagging on covert sabotage capabilities.
1 item
Untrusted advice for AI control: Short, strong advice significantly uplifts weak LLMs
Introduces an untrusted advice protocol where a trusted LLM executes actions and a capable untrusted LLM sends only tiny hints, arguing narrow channels can make AI control near-safe while recovering capability.}