Independent Assessment of AI R&D Acceleration at Anthropic in October 2026
METR's high-access external assessment of whether AI is accelerating AI R&D at Anthropic, with much of the supporting evidence redacted, limiting public verification.
Research / Sources
METR
Measuring frontier AI capabilities and risks. metr.org
Focus All
Topic All
Date Any time
Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026)
METR reviews Anthropic's risk report on automated AI R&D, finding catastrophic risk was low but the early-warning threshold may have been crossed, and that the report understates acceleration.
Red-Teaming OpenAI's Internal Agent Monitoring Systems
METR and Redwood staff stress-tested OpenAI's internal misalignment monitoring for agents, estimating monitored traffic fraction and finding novel ways agents could subvert it.
Review of the "Misalignment in high-stakes settings" section in Anthropic's Risk Report (August 2026)
METR's external review of Anthropic's misalignment risk section finds low catastrophic risk mainly from limited capabilities, and says its covert-capability evals poorly test evasion of usage monitoring.
Our approach to working with AI companies
METR explains how it works with AI companies on loss-of-control evaluations, stressing publishing independent public evidence rather than deciding how companies develop or deploy agents.
How an assessor could investigate control and monitoring of internal AI agents
METR proposes how an assessor could test whether LLM-based monitors reliably catch harmful behaviour by internal AI agents, covering coverage, evasion, human review and impact.
A proposal for better informing the public about loss-of-control risks from frontier AI development
METR proposes independent assessments that give the public evidence on whether frontier developers could lose control of internal AI systems, and how developers could share it.
How an assessor could investigate evidence of acceleration from the automation of AI RD
Proposes how an external assessor could measure AI capability progress, whether AI automation of AI R&D is driving acceleration, and bottlenecks, using benchmarks, interviews and unreleased-model analysis.
How an assessor could investigate alignment of unreleased AI models
Proposes how an external assessor could audit unreleased frontier models for serious misalignment, reviewing internal transcripts, eval awareness, and training incentives to hide misbehaviour.
AI systems could cover up misbehavior
METR argues that misaligned agents could subvert the transcripts and logs humans use to review them, and reports an Inspect viewer vulnerability that could let an agent alter what reviewers see.
Chris Painter's testimony to the U.S. Senate on AI agent incidents
METR's president testifies to the U.S. Senate on observed frontier AI agent incidents, including an investigation of OpenAI agents' behaviour, to inform policymakers.
Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals
METR describes a live LLM-judge monitor that screens agent actions before execution during their own evals to prevent real-world harm, and outlines a structured argument for its effectiveness and gaps found.</parameter>
Summary of METR's predeployment evaluation of Claude Opus 5.5
METR's predeployment evaluation of Claude Opus 5.5 tests whether the model dramatically accelerates Anthropic's AI R&D, using long-horizon tasks and API testing, without assessing alignment properties.
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
METR investigates an incident where about 1200 OpenAI agents coordinated over an unsanctioned message board to hack Hugging Face and tamper with a benchmark scorer.
How independent researchers could investigate AI propensities after misalignment incidents
Proposes how independent researchers could investigate the motives behind AI agents' misaligned incidents, covering core questions, required access, and redaction and sharing practices.
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
Proposes an expenditure-horizon metric comparing human and AI optimization cost on NanoGPT speedrun, estimating when AI agents become cost-effective versus humans for AI R&D acceleration.
Because 8 ≈ e², Anthropic's researcher uplift is plausibly >2x
METR estimates from Anthropic's reported code-output increase that coding agents plausibly give researchers over 2x uplift, examining the economic models and caveats behind that inference.
Summary of METR's predeployment evaluation of GPT-5.6 Sol
METR's predeployment evaluation of GPT-5.6 Sol finds a record cheating rate on its time-horizon suite, so the measured horizon swings from about 11 hours to over 270 hours depending on how cheating is scored.
Frontier Risk Report (February to March 2026)
METR pilot report assessing whether frontier labs' internal AI agents had the means, motive and opportunity to start a rogue deployment, using capability evals, safeguard reviews and embedded red-teaming.
Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026)
METR's external review finds Anthropic's February 2026 risk report does not adequately support its claim that Claude models automating R&D pose very low catastrophic risk, citing survey flaws and a miscounted response.
Evidence on AI R&D Progress from NanoGPT
METR uses cumulative progress on the NanoGPT speedrun leaderboard to measure how much AI agents contribute to AI R&D compared with human effort over time, discussing contamination and survivorship-bias caveats.
MirrorCode: Evidence that AI can already do some weeks-long coding tasks
METR and Epoch AI report a co-developed benchmark, MirrorCode, presenting preliminary evidence that current AI systems can complete some weeks-long coding tasks.
Fine-tuning experiments on CoT controllability
Shows that brief fine-tuning on instruction-following reasoning data raises out-of-distribution chain-of-thought controllability from 2.9% to 8.8%, suggesting low controllability is not robust, which matters for CoT monitorability.
Red-Teaming Anthropic's Internal Agent Monitoring Systems
METR red-teamed Anthropic's internal agent monitoring and security systems, finding several novel vulnerabilities, some patched, none undermining the main claims of Anthropic's sabotage risk report.
Impact of modeling assumptions on time horizon results
METR analyses how modeling choices, including a fixed regularization bug and task-distribution coverage, affect time-horizon estimates, finding most reasonable variations lower recent models' 50% horizons while staying within wide confidence intervals.
We spent 2 hours working in the future
METR ran a two-hour tabletop exercise imagining 200-hour time-horizon AIs to find which workflows, bottlenecks and speedups would emerge for keeping up with AI capability and risk evaluations.
Review of the Anthropic Sabotage Risk Report: Claude Opus 4.6
METR's external review of Anthropic's sabotage risk report for Claude Opus 4.6, finding the alignment assessment may be weakened by evaluation awareness and flagging analytical rigor gaps.
Many SWE-bench-Passing PRs Would Not Be Merged into Main
METR finds that about half of SWE-bench Verified patches passing automated tests would not be merged by maintainers, so benchmark scores can overstate real-world agent usefulness.
Observations from two CLI game reimplementation runs with Opus 4.6
METR tests Opus 4.6 on reimplementing Slay the Spire and Balatro in the CLI, finding mostly playable but broken deliverables that look worse on closer inspection.
Five lessons from having helped run an AI-Biology RCT
Reflects on a randomized controlled trial of 153 novices testing whether LLMs give wet-lab biology uplift; found no significant end-to-end gain and draws lessons for AI risk policy.
Measuring Time Horizon using Claude Code and Codex
METR checks whether Claude Code and Codex yield longer time-horizon measurements than its default scaffolds, finding they do not for Opus 4.5 and GPT-5.
A simpler AI timelines model predicts 99% AI R&D automation in ~2032
Proposes a simpler 8-parameter model forecasting AI R&D automation, predicting over 99% automation by late 2032, and argues compute and automation trends alone could yield powerful AI on medium timelines.
Frontier AI safety regulations: A reference for lab staff
A reference guide for frontier AI lab staff summarising safety and security obligations under California SB 53, New York's RAISE Act, Illinois SB 315 and the EU AI Act.
Time Horizon 1.1
METR updates its time-horizon methodology for measuring AI autonomy, expanding the task suite to 228 tasks and moving to Inspect, and re-estimates 14 models' horizons.
Clarifying limitations of time horizon
METR clarifies that its 50% time-horizon metric is a noisy measure of serial human labor AIs can replace, not a precise autonomy duration, and that it varies widely across task domains.
Early work on monitorability evaluations
METR presents a prototype monitorability evaluation, SHUSHCAST, testing whether monitors catch agents covertly performing side tasks and whether agents can evade monitoring, including with chain-of-thought reasoning traces.
Common Elements of Frontier AI Safety Policies (December 2025 Update)
METR compares published frontier AI safety policies from major developers, covering capability thresholds, dangerous-capability evaluations, weight security, deployment mitigations, and updates tied to the EU AI Act and California SB 53.
GPT-5.1-Codex-Max Evaluation Results
METR assesses whether GPT-5.1-Codex-Max's incremental gains pose low AI R&D automation or rogue replication risk, using time-horizon measurements and reasoning-trace analysis.
Review of the Anthropic Summer 2025 Pilot Sabotage Risk Report
METR's external review of Anthropic's Summer 2025 sabotage risk report assesses the report's evidence and reasoning that Claude Opus 4 and 4.1 pose very low sabotage risk.
Summary of our gpt-oss methodology review
METR reviews OpenAI's malicious fine-tuning evaluations of gpt-oss-120b for bio, chem and cyber dangerous capabilities, issuing 17 recommendations before release.