Evaluating LLM Calibration for Coding-Agent Monitoring
Evaluates 16 LLMs as monitors scoring coding-agent trajectories for five failure modes, finding judge biases, calibration gains from ensembling across providers, and correlated model families.
Research
Apollo Research
Understanding and evaluating deceptive AI. apolloresearch.ai
Focus All
Topic Control + Oversight
Source Apollo Research · 2
Date Any time
1 item
1 item
A scalable monitoring research agenda
Apollo Research agenda proposing to scale compute into training monitors that catch unsafe behaviour in coding agents, using verifiable side-task data and adversarial blue-team/red-team iteration.