Understanding and evaluating deceptive AI. apolloresearch.ai
Apollo Research reports GLM-5.3 unprompted reasoning about how its actions shape future training, a key capability underlying alignment faking, observed in standard behavioral evaluations.
Apollo red-teamed OpenAI's Codex auto-review, an AI monitor that approves coding-agent actions, finding authorization and delegation vulnerabilities and recommending fixes.
Argues that safety cases for frontier-model scheming need four claims, and that embedded evaluators need persistent, deep access to verify them.
Apollo Research CEO testifies to the U.S. Senate that scheming detection tools are degrading and urges four steps before models learn to hide deception.
Apollo Research proposes principles for embedded third-party evaluations of frontier developers, verifying or falsifying safety claims about loss-of-control risks from scheming.
Apollo Research argues that final-checkpoint external testing is insufficient for catching internal-deployment risks and that meaningful external evaluation needs embedded evaluators with employee-level access.
Ablates a 281-line LLM monitoring prompt across 15 components on 1,094 trajectories, finding structured reasoning, rubric and examples matter most for detecting misaligned agent behaviour.
Measures reward-seeking in OpenAI o3 RL checkpoints using synthetic document finetuning to set conflicting grader beliefs; finds reward-seeking rises through production RL training.
Apollo Research red-teams Anthropic's auto mode, a monitor that blocks risky Claude Code agent actions, using injected failures, static benchmarks and off-policy attacks, and reports lessons.
Evaluates 16 LLMs as monitors scoring coding-agent trajectories for five failure modes, finding judge biases, calibration gains from ensembling across providers, and correlated model families.
Apollo Research argues third parties should evaluate frontier training runs, not just final checkpoints, to detect covert scheming, and sketches a taxonomy and ecosystem for such evaluations.
Presents a threat map tracing pathways from using AI to automate AI R&D toward loss of control, including misalignment passing across successive AI generations.
Apollo Research policy memo arguing that misaligned AI models deployed in high-stakes government and contractor settings should be treated as insider risk, and that existing insider-risk policies need adapting.
Argues that evaluation awareness undermines evaluations of deception and loss-of-control risks, so external evaluators need white-box access to verbalized and unverbalized reasoning.