Evaluating LLM Calibration for Coding-Agent Monitoring
Evaluates 16 LLMs as monitors scoring coding-agent trajectories for five failure modes, finding judge biases, calibration gains from ensembling across providers, and correlated model families.
Research / Sources
Apollo Research
Understanding and evaluating deceptive AI. apolloresearch.ai
Focus All
Topic Oversight
Date Any time
A scalable monitoring research agenda
Apollo Research agenda proposing to scale compute into training monitors that catch unsafe behaviour in coding agents, using verifiable side-task data and adversarial blue-team/red-team iteration.