On Testifying on Misaligned AI in the U.S. Senate
Apollo Research CEO testifies to the U.S. Senate that scheming detection tools are degrading and urges four steps before models learn to hide deception.
Research / Sources
Apollo Research
Understanding and evaluating deceptive AI. apolloresearch.ai
Focus All
Topic Evals & standards
Date Any time
Principles for Embedded Evaluations
Apollo Research proposes principles for embedded third-party evaluations of frontier developers, verifying or falsifying safety claims about loss-of-control risks from scheming.
Embedded Evaluators are necessary for meaningful external testing
Apollo Research argues that final-checkpoint external testing is insufficient for catching internal-deployment risks and that meaningful external evaluation needs embedded evaluators with employee-level access.
An Overview of Our Current Governance Efforts
Apollo Research's governance team overview of its workstreams on governing internal deployment of advanced AI, EU AI Act policy, and scheming and dangerous-capability evaluation policy.
NIST RFI: Security Considerations for AI Agents
Apollo Research's comment to NIST on agent security recommends scheming and misalignment evaluations, AI control and monitoring, and tighter oversight of high-stakes agentic deployments to prevent loss of control.
Our Norms on Security, Science Communication and Conflicts of Interest
Apollo Research describes its internal norms on conflicts of interest, evaluator compensation, publishing, and security for its AI safety evaluation work. It is organisational policy rather than a technical safety result.
Assurance of Frontier AI Built for National Security
Policy memo giving US national-security agencies a roadmap to use scheming and control evaluations within AI assurance frameworks, treating scheming as a sign of low reliability and governability.
Capturing and Countering Threats to National Security: a Blueprint for an Agile AI Incident Regime
Proposes a three-part AI incident regime modeled on US incident response, so states can track national security threats from frontier AI and swiftly contain them.
Forecasting Frontier Language Model Agent Capabilities
Compares six methods for forecasting frontier LM agent benchmark scores ahead of time, using Chatbot Arena Elo and compute, backtested on OpenLLM v2 models, to anticipate dangerous capabilities on SWE-Bench, Cybench and RE-Bench.
Precursory Capabilities: A Refinement to Pre-deployment Information Sharing and Tripwire Capabilities
Proposes "precursory capabilities" and a zoning taxonomy so pre-deployment information sharing and tripwire thresholds can flag frontier models approaching catastrophic-risk red lines.
The Evals Gap
Apollo Research argues that high-quality frontier-model safety evals are scarce and slow to build, and calls for more resources to close this "evals gap."
Towards Safety Cases For AI Scheming
Sketches evaluation-based safety case arguments that frontier AI systems are unlikely to cause catastrophic harm through covert scheming, and notes the current framework lacks crucial details.
Our Current Policy Positions
Apollo Research outlines five policy positions for governments, arguing evaluations alone are insufficient and that AI safety needs defence-in-depth, independent verification, and interpretability.
Our Work Advancing Scientific Understanding To Foster An Effective International Evaluation Ecosystem
Apollo Research announces its engagement with US AI governance bodies (AISI Consortium, NIST working groups) to build an international evaluation ecosystem for dangerous capabilities and misalignment.
We Need A ‘Science of Evals’
Argues AI safety evals need a rigorous "Science of Evals" for trustworthy methodology, citing prompt-sensitivity problems that undermine eval results used in safety and deployment decisions.
A Starter Guide For Evals
An Apollo Research starter guide introducing model evaluations: what evals measure, why they inform decisions like Responsible Scaling Policies, and how to start a career in the field.
Theories of Change for AI Auditing
Apollo Research lays out how AI auditing, including system evaluations and training audits, could reduce catastrophic misalignment risk, and where such approaches may fail.
A Causal Framework for AI Regulation and Auditing
Proposes a causal framework for auditing and regulating AI systems to assure responsible development, focusing on catastrophic risks and the role of available affordances in designing oversight.
The UK AI Safety Summit: Our Recommendations
Apollo Research recommends that the UK AI Safety Summit pursue international commitments on AI risk classification, responsibilities across the value chain, and evaluation-based compliance for frontier models.
Security at Apollo Research
Apollo Research describes its security norms for handling lab IP and model access while auditing frontier AI for deception, noting the policy is early-stage.