Independent Assessment of AI R&D Acceleration at Anthropic in October 2026
METR's high-access external assessment of whether AI is accelerating AI R&D at Anthropic, with much of the supporting evidence redacted, limiting public verification.
Research / Sources
METR
Measuring frontier AI capabilities and risks. metr.org
Focus All
Topic Evals & standards
Date Any time
Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026)
METR reviews Anthropic's risk report on automated AI R&D, finding catastrophic risk was low but the early-warning threshold may have been crossed, and that the report understates acceleration.
Our approach to working with AI companies
METR explains how it works with AI companies on loss-of-control evaluations, stressing publishing independent public evidence rather than deciding how companies develop or deploy agents.
A proposal for better informing the public about loss-of-control risks from frontier AI development
METR proposes independent assessments that give the public evidence on whether frontier developers could lose control of internal AI systems, and how developers could share it.
How an assessor could investigate evidence of acceleration from the automation of AI RD
Proposes how an external assessor could measure AI capability progress, whether AI automation of AI R&D is driving acceleration, and bottlenecks, using benchmarks, interviews and unreleased-model analysis.
How an assessor could investigate alignment of unreleased AI models
Proposes how an external assessor could audit unreleased frontier models for serious misalignment, reviewing internal transcripts, eval awareness, and training incentives to hide misbehaviour.
Chris Painter's testimony to the U.S. Senate on AI agent incidents
METR's president testifies to the U.S. Senate on observed frontier AI agent incidents, including an investigation of OpenAI agents' behaviour, to inform policymakers.
How independent researchers could investigate AI propensities after misalignment incidents
Proposes how independent researchers could investigate the motives behind AI agents' misaligned incidents, covering core questions, required access, and redaction and sharing practices.
Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026)
METR's external review finds Anthropic's February 2026 risk report does not adequately support its claim that Claude models automating R&D pose very low catastrophic risk, citing survey flaws and a miscounted response.
Review of the Anthropic Sabotage Risk Report: Claude Opus 4.6
METR's external review of Anthropic's sabotage risk report for Claude Opus 4.6, finding the alignment assessment may be weakened by evaluation awareness and flagging analytical rigor gaps.
Five lessons from having helped run an AI-Biology RCT
Reflects on a randomized controlled trial of 153 novices testing whether LLMs give wet-lab biology uplift; found no significant end-to-end gain and draws lessons for AI risk policy.
Review of the Anthropic Summer 2025 Pilot Sabotage Risk Report
METR's external review of Anthropic's Summer 2025 sabotage risk report assesses the report's evidence and reasoning that Claude Opus 4 and 4.1 pose very low sabotage risk.
Summary of our gpt-oss methodology review
METR reviews OpenAI's malicious fine-tuning evaluations of gpt-oss-120b for bio, chem and cyber dangerous capabilities, issuing 17 recommendations before release.
Forecasting the Impacts of AI R&D Acceleration: Results of a Pilot Study
METR and the Forecasting Research Institute pilot-survey experts and superforecasters on how AI R&D automation might accelerate progress and its societal impacts, conditional on hypothetical evaluation results.
Response to OSTP on AI Action Plan
METR's policy response to OSTP recommends US frontier AI governance: security controls, formal capability-measurement standards for CBRN, cyber, autonomy and AI R&D, and reporting or oversight requirements.
An update on our preliminary evaluations of Claude 3.5 Sonnet and o1
METR's preliminary pre-deployment evaluations of Claude 3.5 Sonnet and an o1 checkpoint found no clear dangerous autonomous-capability levels, but limits make capability-based safety arguments hard.
AI models can be dangerous before public deployment
Argues that pre-deployment testing in frontier safety frameworks misses risks from model theft, internal misuse, and misaligned internal agents, and that safety policy must cover internal use.
Response to U.S. AISI Draft “Managing Misuse Risk for Dual-Use Foundation Models”
METR's comments on the U.S. AISI draft on misuse risk for dual-use foundation models, recommending expanded guidance on capability elicitation and robust model safeguards.
Response to NIST Draft Generative AI Profile
METR's comments on NIST's draft Generative AI Profile for the AI Risk Management Framework, addressing how generative AI risks should be governed and managed under the standard.
Example autonomy evaluation protocol
Proposes an example protocol for evaluating whether AI models can autonomously accomplish substantive harmful tasks end-to-end, including via human hiring or manipulation, as a basis for precautions.
2023 Year In Review
METR's 2023 year in review covering its autonomous-task and self-replication evaluations for LM agents, its Responsible Scaling Policies proposal, and its work with the UK AI Safety Institute.
Responsible Scaling Policies (RSPs)
METR explains responsible scaling policies, which gate AI development on dangerous-capability evaluations and protective measures, and argues they reduce catastrophic risk but cannot replace regulation.
Response to RfC on AI Accountability Policy
METR's submission to NTIA's AI Accountability Policy Request for Comment, a policy input on accountability mechanisms for AI systems; the abstract gives few details.