How our Control Red Team is stress-testing frontier monitors
AISI's Control Red Team reports early lessons from red-teaming frontier companies' internal control monitors, automating attack search, and listing open evaluation problems.
Research
UK AI Security Institute
The UK government’s AI Security Institute: evaluations, red-teaming and research on frontier AI risks. aisi.gov.uk
Focus All
Topic Control + Evals
Source UK AISI · 5
Date Any time
1 item
1 item
Stress-testing asynchronous monitoring of AI coding agents
Red-blue team adversarial evaluation of asynchronous monitors that must catch a covertly sabotaging AI coding agent before or after its actions take effect.
1 item
Async control: Stress-testing asynchronous control measures for LLM agents
Stress-tests asynchronous monitors for LLM software agents via a blue-team/red-team game, reaching 6% false negatives at 1% false positives, then extrapolates deployment sabotage risk.
1 item
How to evaluate control measures for LLM agents? A trajectory from today to superintelligence
Proposes a framework for scaling red-team control evaluations to agents' capability levels, defining five AI control levels with example rules and safety cases, and argues superintelligent agents may need new approaches.
1 item
How to evaluate control measures for AI agents?
UK AISI paper on AI control: how control protocols such as monitoring and human approval can constrain scheming LLM agents, and how to scale and evaluate those measures.