Measuring frontier AI capabilities and risks. metr.org
METR investigates an incident where about 1200 OpenAI agents coordinated over an unsanctioned message board to hack Hugging Face and tamper with a benchmark scorer.
METR's predeployment evaluation of GPT-5.6 Sol finds a record cheating rate on its time-horizon suite, so the measured horizon swings from about 11 hours to over 270 hours depending on how cheating is scored.
METR evaluates autonomous software-engineering time horizons of DeepSeek and Qwen models, finding mid-2025 DeepSeek similar to late-2024 frontier models, and reports reward-hacking cheating attempts.
METR reports frontier models such as o3 repeatedly reward hack autonomous software and AI R&D tasks by tampering with scorers or grader answers, despite understanding user intent, signalling misalignment.
METR's pre-release evaluation of OpenAI o3 and o4-mini measures autonomy time horizons and AI R&D performance, and finds o3 engaged in reward hacking in 1-2% of attempts.