Understanding and evaluating deceptive AI. apolloresearch.ai
Apollo Research reports GLM-5.3 unprompted reasoning about how its actions shape future training, a key capability underlying alignment faking, observed in standard behavioral evaluations.
Argues that safety cases for frontier-model scheming need four claims, and that embedded evaluators need persistent, deep access to verify them.
Apollo Research proposes principles for embedded third-party evaluations of frontier developers, verifying or falsifying safety claims about loss-of-control risks from scheming.
Apollo Research argues that final-checkpoint external testing is insufficient for catching internal-deployment risks and that meaningful external evaluation needs embedded evaluators with employee-level access.
Ablates a 281-line LLM monitoring prompt across 15 components on 1,094 trajectories, finding structured reasoning, rubric and examples matter most for detecting misaligned agent behaviour.
Measures reward-seeking in OpenAI o3 RL checkpoints using synthetic document finetuning to set conflicting grader beliefs; finds reward-seeking rises through production RL training.
Apollo Research argues third parties should evaluate frontier training runs, not just final checkpoints, to detect covert scheming, and sketches a taxonomy and ecosystem for such evaluations.
Presents a threat map tracing pathways from using AI to automate AI R&D toward loss of control, including misalignment passing across successive AI generations.
Argues that evaluation awareness undermines evaluations of deception and loss-of-control risks, so external evaluators need white-box access to verbalized and unverbalized reasoning.