Reflections from red-teaming Codex auto-review
Apollo red-teamed OpenAI's Codex auto-review, an AI monitor that approves coding-agent actions, finding authorization and delegation vulnerabilities and recommending fixes.
Research
Apollo Research
Understanding and evaluating deceptive AI. apolloresearch.ai
Focus All
Topic Control
Source Apollo Research · 4
Date Past 6 months
1 item
1 item
What makes a good monitoring prompt?
Ablates a 281-line LLM monitoring prompt across 15 components on 1,094 trajectories, finding structured reasoning, rubric and examples matter most for detecting misaligned agent behaviour.
1 item
Red-teaming auto mode: lessons from our first external monitor campaign with Anthropic
Apollo Research red-teams Anthropic's auto mode, a monitor that blocks risky Claude Code agent actions, using injected failures, static benchmarks and off-policy attacks, and reports lessons.
1 item
Evaluating LLM Calibration for Coding-Agent Monitoring
Evaluates 16 LLMs as monitors scoring coding-agent trajectories for five failure modes, finding judge biases, calibration gains from ensembling across providers, and correlated model families.