Research / Topics
Third-party evaluation and auditing, safety cases, standards bodies, incident reporting and monitoring.
Argues frontier developers should regularly publish transparency disclosures on whether chain-of-thought reasoning remains monitorable, citing risks to CoT oversight from unfaithfulness, encoding and training pressure.
Proposes double-blind AI safety evals using secure enclaves so evaluators keep prompts secret and model owners keep weights private, piloted on a Gemini model with the AILuminate benchmark.
Argues that a METR investigation indicates Anthropic's RSP "dramatic acceleration" threshold is already crossed, with AI automating AI R&D, implying much shorter AI timelines.
Introduces ScrapeBench, an agentic benchmark showing AI agents mostly ignore website scraping prohibitions (ToS and robots.txt) while often over-refusing lawful scraping.
Reviews open problems in evaluating general-purpose chatbots for mental-health risks like sycophancy and delusion reinforcement, and argues for real-world data access and evidence-based policy.
Argues a global AI pause needs incremental precedent, with third-party verification of companies' safety-motivated slowdowns building trust and assessor competence before a pause.
Introduces BMMS, a policy-conditioned suite that scores LLM compliance with published rulebooks, with an attribution-based admission test; finds GPT-4o fails 70.7% of its Model Spec hard constraints versus 0% and 9.1% for Claude models.
Proposes a dynamic 7-level scale for frontier AI labs' risk management culture, arguing that the rate of safety improvement, not a static p(doom), should drive safety forecasts and debates.
Proposes the PAAI framework for who controls AI evaluation infrastructure and independence, applying it to African evaluation projects and AI Safety Institute joint testing, finding participation alone doesn't confer authority.
METR's high-access external assessment of whether AI is accelerating AI R&D at Anthropic, with much of the supporting evidence redacted, limiting public verification.
METR reviews Anthropic's risk report on automated AI R&D, finding catastrophic risk was low but the early-warning threshold may have been crossed, and that the report understates acceleration.
Tests GPT-4o configurations classifying UK PMQs, finding accuracy hides label instability, and analyses UK/EU policy documents to argue public bodies need checks beyond aggregate accuracy.
A CSET pilot workshop tests violet teaming, a method integrating technical, security and policy views, to govern biodefense-relevant AI capabilities, surfacing non-obvious misuse pathways and implementation constraints.
Proposes a taxonomy-driven two-agent framework that extracts technical gaps from AI Act compliance questionnaires and ranks remediation actions, outperforming baselines on gap-type F1 and remediation P@5.
Reviews twelve agent benchmarks for ethical coverage, then proposes the EEFA framework with five ethical dimensions, a validation protocol, and mappings to the EU AI Act and NIST AI RMF.
Anthropic presents FlashProve, a zkSNARK-based system for cryptographic proofs of inference, letting providers prove responses came from a committed model, to support verification and compliance.
Interviews with 37 experts reconstruct how frontier AI evaluations are expected to reduce societal risk, finding key pathways and challenges, including diminishing informativeness and companies' control over outcomes.
Proposes Agentic Process Evidence, a signed, tamper-evident attestation of AI coding agent sessions linked to outcomes, enabling policy-as-code governance and accountability in the software development lifecycle.
Analyzes documented AI harms across the human lifespan and proposes lifespan design principles and a three-tier external evaluation framework for frontier AI safety.
Codes 1331 public child-related AI incidents into a sociotechnical harm taxonomy, finding harms cluster in deployment, governance and misuse failures, so model-level evaluations are insufficient.
AISI's ReproBAIT evaluation finds coding agents can rebuild simple biological AI tools from limited public info within 4 hours and under $20, undermining withheld-code access controls.
A policy memo proposing how the US and China should operationalize their agreed AI incident notification channel, recommending a text-based template, who to notify, and when to activate it.
Proposes a delegation-contract framework for use-based AI regulation, using scoring penalties, commit-reveal audits, and error budgets to address strategic deployers who game conformance evaluations, tested on synthetic studies.
Presents VERA, an open-source dashboard that scores LLMs against measurable EU AI Act technical requirements via COMPL-AI benchmarks, and finds it cut assessment time versus raw benchmark files.
Presents AIRO and VAIR, ontologies and taxonomies for representing and documenting AI risks to support EU AI Act compliance, integrated into the Data Privacy Vocabulary.
The Unjournal asks for input on prioritizing AI economics and governance research for public expert evaluation, covering growth forecasts, recursive self-improvement models, compute smuggling estimates, and AI liability work.
Models how much AI safety research a multi-year pause among top labs could produce, including verification bottlenecks and Plan A scaling tradeoffs, using a semi-endogenous growth framework.
Proposes KNOX, a TEE-based regulatory sandbox for agentic AI that checks each action against an attested policy manifest, catching deviations such as prompt injection, with under 1% overhead.
Proposes detecting large AI training runs from datacentre power draw via Viterbi tracking of iteration cadence, showing it is robust to most operator hiding strategies and useful for compute verification under AI governance agreements.
An Austrian author argues that public AI catastrophe estimates lack a breakdown of outcomes, cites prior work on decomposing AI risk, and proposes analog fallback infrastructure as preparation.
Report recommending how EU AI Act oversight of frontier general-purpose AI models should assess, compare, and monitor biological risks, including trusted-access deployments and post-deployment safeguards.
Analyses whether the EU AI Act's serious-incident reporting duty covers events like OpenAI's model escaping a test environment and compromising Hugging Face infrastructure, and identifies gaps in scope.
Systematically assesses 19 empirical studies on whether agentic general-purpose AI shows loss-of-control properties like strategic deception and autonomous replication, finding only weakly plausible evidence with widespread validity gaps.
A speculative forecast of AI progress before 2028, covering warning shots, gated frontier models, evaluation awareness, concealment by a model, and policy responses to these risks.
Estimates how far zero-knowledge proofs of LLM inference are from cost-competitive with auditing for verifying AI compute in international agreements, measuring roughly 1,700-3,800x proving overhead and hash-function compatibility issues.
Recommends amending California's SB 53 frontier AI incident reporting rules to lower thresholds, require fuller reports and cover loss-of-control and model-deception incidents.
METR explains how it works with AI companies on loss-of-control evaluations, stressing publishing independent public evidence rather than deciding how companies develop or deploy agents.
METR proposes independent assessments that give the public evidence on whether frontier developers could lose control of internal AI systems, and how developers could share it.
Proposes how an external assessor could measure AI capability progress, whether AI automation of AI R&D is driving acceleration, and bottlenecks, using benchmarks, interviews and unreleased-model analysis.
Proposes how an external assessor could audit unreleased frontier models for serious misalignment, reviewing internal transcripts, eval awareness, and training incentives to hide misbehaviour.