Research / Sources
Measuring frontier AI capabilities and risks. metr.org
METR proposes how an assessor could test whether LLM-based monitors reliably catch harmful behaviour by internal AI agents, covering coverage, evasion, human review and impact.
METR argues that misaligned agents could subvert the transcripts and logs humans use to review them, and reports an Inspect viewer vulnerability that could let an agent alter what reviewers see.
METR investigates an incident where about 1200 OpenAI agents coordinated over an unsanctioned message board to hack Hugging Face and tamper with a benchmark scorer.
METR (formerly ARC Evals) introduces a methodology for testing whether LLM agents can autonomously acquire resources, replicate and adapt, using 12 realistic tasks and example scaffolded agents as an early-warning pilot.