Red-Teaming OpenAI's Internal Agent Monitoring Systems
METR and Redwood staff stress-tested OpenAI's internal misalignment monitoring for agents, estimating monitored traffic fraction and finding novel ways agents could subvert it.
Research / Sources
METR
Measuring frontier AI capabilities and risks. metr.org
Focus All
Topic Control
Date Any time
Review of the "Misalignment in high-stakes settings" section in Anthropic's Risk Report (August 2026)
METR's external review of Anthropic's misalignment risk section finds low catastrophic risk mainly from limited capabilities, and says its covert-capability evals poorly test evasion of usage monitoring.
How an assessor could investigate control and monitoring of internal AI agents
METR proposes how an assessor could test whether LLM-based monitors reliably catch harmful behaviour by internal AI agents, covering coverage, evasion, human review and impact.
AI systems could cover up misbehavior
METR argues that misaligned agents could subvert the transcripts and logs humans use to review them, and reports an Inspect viewer vulnerability that could let an agent alter what reviewers see.
Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals
METR describes a live LLM-judge monitor that screens agent actions before execution during their own evals to prevent real-world harm, and outlines a structured argument for its effectiveness and gaps found.</parameter>
Frontier Risk Report (February to March 2026)
METR pilot report assessing whether frontier labs' internal AI agents had the means, motive and opportunity to start a rogue deployment, using capability evals, safeguard reviews and embedded red-teaming.
Fine-tuning experiments on CoT controllability
Shows that brief fine-tuning on instruction-following reasoning data raises out-of-distribution chain-of-thought controllability from 2.9% to 8.8%, suggesting low controllability is not robust, which matters for CoT monitorability.
Red-Teaming Anthropic's Internal Agent Monitoring Systems
METR red-teamed Anthropic's internal agent monitoring and security systems, finding several novel vulnerabilities, some patched, none undermining the main claims of Anthropic's sabotage risk report.
Early work on monitorability evaluations
METR presents a prototype monitorability evaluation, SHUSHCAST, testing whether monitors catch agents covertly performing side tasks and whether agents can evade monitoring, including with chain-of-thought reasoning traces.
MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity
METR releases MALT, a labeled dataset of natural and prompted agent transcripts showing reward hacking and sandbagging, to validate monitors that check evaluation integrity.
Early Results on Monitorability in QA Settings
METR studies how often agents can fool a monitor in QA side-task settings, finding evasion falls as side tasks get harder and that a small 8b agent can be trained to evade GPT-4o.
Claude, GPT, and Gemini All Struggle to Evade Monitors
Replicates a DeepMind chain-of-thought monitorability experiment across Claude, GPT-4o and Gemini models, finding that most actors struggle to hide reasoning from monitors on hard tasks.
CoT May Be Highly Informative Despite “Unfaithfulness”
Argues CoT need not be fully faithful to be useful for safety, and tests whether clue-using reasoning in chains of thought can be reliably detected by replicating Claude system card evaluations with harder clues.
Why it’s good for AI reasoning to be legible and faithful
METR argues that reasoning models' chain-of-thought should be legible and faithful, since this aids detecting mistakes, hidden agendas, and deception before harm occurs.