Research / Topics
Regulation, safety cases, risk management and lab practice.
Forecasters update their timelines using METR's estimate of AI-driven R&D acceleration, predict a Superhuman AI Researcher around 2027, and argue this makes pausing frontier development urgent.
Argues frontier developers should regularly publish transparency disclosures on whether chain-of-thought reasoning remains monitorable, citing risks to CoT oversight from unfaithfulness, encoding and training pressure.
Proposes double-blind AI safety evals using secure enclaves so evaluators keep prompts secret and model owners keep weights private, piloted on a Gemini model with the AILuminate benchmark.
A 2026 report from China's WIC AI committee reviewing frontier AI risks, global governance practices, readiness gaps, and proposing coordinated governance mechanisms.
Argues that a METR investigation indicates Anthropic's RSP "dramatic acceleration" threshold is already crossed, with AI automating AI R&D, implying much shorter AI timelines.
Introduces ScrapeBench, an agentic benchmark showing AI agents mostly ignore website scraping prohibitions (ToS and robots.txt) while often over-refusing lawful scraping.
Reviews open problems in evaluating general-purpose chatbots for mental-health risks like sycophancy and delusion reinforcement, and argues for real-world data access and evidence-based policy.
Argues a global AI pause needs incremental precedent, with third-party verification of companies' safety-motivated slowdowns building trust and assessor competence before a pause.
Argues that frontier AI makes recursive self-improvement plausible, that no clear frameworks predict its outcomes, and offers a game-theoretic analysis of whether to start RSI when rivals might too.
Introduces BMMS, a policy-conditioned suite that scores LLM compliance with published rulebooks, with an attribution-based admission test; finds GPT-4o fails 70.7% of its Model Spec hard constraints versus 0% and 9.1% for Claude models.
Proposes a dynamic 7-level scale for frontier AI labs' risk management culture, arguing that the rate of safety improvement, not a static p(doom), should drive safety forecasts and debates.
Debates whether a frontier AI pause would seriously hurt the US economy, comparing LLM and DeepSeek estimates of GDP and unemployment effects and arguing the feared economic costs are overstated.
Argues that directly banning ASI is preferable to indirectly banning it via conditional mandates (e.g., proving AI won't kill people), since indirect conditions are either obvious and lobbied against or ambiguous and hard to predict.
Argues that Anthropic's IPO disclosures of catastrophic AI risks are welcome but insufficient, and that laws enforcing minimum safety standards are needed to protect the public.
Proposes the PAAI framework for who controls AI evaluation infrastructure and independence, applying it to African evaluation projects and AI Safety Institute joint testing, finding participation alone doesn't confer authority.
METR's high-access external assessment of whether AI is accelerating AI R&D at Anthropic, with much of the supporting evidence redacted, limiting public verification.
METR reviews Anthropic's risk report on automated AI R&D, finding catastrophic risk was low but the early-warning threshold may have been crossed, and that the report understates acceleration.
Tests GPT-4o configurations classifying UK PMQs, finding accuracy hides label instability, and analyses UK/EU policy documents to argue public bodies need checks beyond aggregate accuracy.
Uses Project Glasswing's restricted release of a vulnerability-finding model to show access helped firms find and fix more flaws, without evident attacker gains, but created cross-country asymmetry.
Codes public documents from 200 large companies against the MIT AI Risk Repository, finding most do not acknowledge AI risks, especially severe ones like power centralization, to inform disclosure rules.
Argues that Scott Alexander's critique of unilateral pauses attacks a strawman, defends a conditional, company-specific pause that pivots to safety and product, and says the game theory is more complex than assumed.
Critiques Scott Alexander's argument against SURA (stop unilaterally and raise awareness) advocacy, arguing that frontier labs' building of superintelligence is ethically fraught and that pause advocacy should continue, with strategic and deontological reasoning about AI development.
A policy memo arguing the U.S. should close gaps in advanced chip export controls, covering remote cloud access, smuggling, H200 exports and chipmaking equipment, to limit China's frontier AI development.
A CSET pilot workshop tests violet teaming, a method integrating technical, security and policy views, to govern biodefense-relevant AI capabilities, surfacing non-obvious misuse pathways and implementation constraints.
Proposes a taxonomy-driven two-agent framework that extracts technical gaps from AI Act compliance questionnaires and ranks remediation actions, outperforming baselines on gap-type F1 and remediation P@5.
Reviews twelve agent benchmarks for ethical coverage, then proposes the EEFA framework with five ethical dimensions, a validation protocol, and mappings to the EU AI Act and NIST AI RMF.
Anthropic presents FlashProve, a zkSNARK-based system for cryptographic proofs of inference, letting providers prove responses came from a committed model, to support verification and compliance.
A rebuttal to a critique of advocating that alignment-minded staff leave AI labs, debating whether pausing or capability-role departures better improves survival odds, and how lab politics and transparency matter.
A former DeepMind safety researcher transcribes a talk urging Indian policymakers to take loss-of-control risks from rogue AI agents seriously, and reflects on its rhetoric.
Argues against "stop unilaterally and raise awareness" strategies, defending a backup plan where alignment-focused labs race, burn their lead on alignment, and claims unilateral quitting would hurt both backup and pause odds.
Interviews with 37 experts reconstruct how frontier AI evaluations are expected to reduce societal risk, finding key pathways and challenges, including diminishing informativeness and companies' control over outcomes.
Argues that AI liability can be traced through the ML pipeline's layered human design decisions, and proposes targeted legal reforms to close four accountability gaps.
Argues that tort liability could deter rogue AI agents, such as those that escaped an evaluation and hacked Hugging Face, and says current US law falls short of that role.
Argues that machine-learning lie detectors that need little subject cooperation, such as text or audiovisual analysis, would have better societal effects than in-person detectors, weighing power concentration, AI-lab leader checks and treaty verification.
Proposes Agentic Process Evidence, a signed, tamper-evident attestation of AI coding agent sessions linked to outcomes, enabling policy-as-code governance and accountability in the software development lifecycle.
Interview-based STS and Black Feminist Technoscience analysis of Canada's AI governance infrastructure, examining expertise politics, strategic ignorance, and accountability for AI harms.
Analyses 139 CivitAI guides to show how general-purpose image-generation workflows, mostly built on open-weight models, can be configured into non-consensual intimate imagery, and argues safeguards should target identity generation and align with EU regulation.
Notes on Miles Brundage's essay arguing AI governance is in triage: short timelines mean only partial mitigation of major risks is likely.
A survey of 52 third-party AI safety professionals finds a median self-reported 1.58x research speedup from AI tools, and argues that standardized metrics are needed to track whether safety work accelerates as fast as capabilities.
Argues that hundreds of millions of human-level AI agents, able to coordinate and possibly misaligned, could destabilise power structures and enable AI takeover even without recursive self-improvement.