How our Control Red Team is stress-testing frontier monitors
AISI's Control Red Team reports early lessons from red-teaming frontier companies' internal control monitors, automating attack search, and listing open evaluation problems.
Research
UK AI Security Institute
The UK government’s AI Security Institute: evaluations, red-teaming and research on frontier AI risks. aisi.gov.uk
Focus All
Topic Control
Source UK AISI · 19
Date Any time
1 item
1 item
Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors
Studies AI control against distributed multi-agent attacks, finding per-agent monitors catch fewer attackers as agents coordinate, and that stronger trusted monitors reduce undetected attacks.
1 item
"Did you lie?": Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms
Evaluates lie detectors (CoT judge, logprob classifier, activation probes, new Did-You-Lie probes) on belief-verified model organisms and 31 models; activation methods degrade on trained organisms while CoT judge stays strong.
2 items
Will it become harder to oversee AI systems?
Report mapping AI oversight methods (audits, monitoring, incident investigation) across internal activations, chain-of-thought, actions and inter-agent channels, and how they may degrade as capabilities advance.
Loss of Oversight: How AI systems may become harder to audit, monitor, and investigate
Interviews and literature review assess how five oversight signals (behaviour, chain-of-thought, internals, memory, honesty training) may degrade, and which technical levers could preserve them.
1 item
A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring
Proposes a decision-theoretic formalisation of steganography and a "steganographic gap" metric to detect and quantify hidden reasoning in LLMs that could evade monitoring.
1 item
Stress-testing asynchronous monitoring of AI coding agents
Red-blue team adversarial evaluation of asynchronous monitors that must catch a covertly sabotaging AI coding agent before or after its actions take effect.
2 items
Practical challenges of control monitoring in frontier AI deployments
Analyses design choices for control monitors of AI agents in real deployments (synchronous, semi-synchronous, asynchronous), identifies oversight, latency and recovery challenges, and uses a safety case sketch.
Async control: Stress-testing asynchronous control measures for LLM agents
Stress-tests asynchronous monitors for LLM software agents via a blue-team/red-team game, reaching 6% false negatives at 1% false positives, then extrapolates deployment sabotage risk.
1 item
Deepening our partnership with Google DeepMind
Announces a UK AISI–Google DeepMind research partnership with planned joint work on chain-of-thought monitoring, socio-affective alignment, and AI's economic impact. No results are reported.
1 item
Introducing ControlArena: A library for running AI control experiments
Introduces ControlArena, an open library of shared settings, main and side tasks, and control protocols for making AI control experiments easy, consistent, and repeatable.
1 item
Lessons from studying two-hop latent reasoning
Fine-tunes LLMs on synthetic facts to test whether two-hop reasoning happens latently without chain-of-thought; finds models compose a synthetic and a natural fact but not two synthetic facts.
1 item
Chain of thought monitorability: A new and fragile opportunity for AI safety
Position piece arguing chain-of-thought monitoring offers a fragile but valuable way to catch AI misbehaviour, and recommending further research and developer attention to preserving CoT monitorability.
2 items
White Box Control at UK AISI - Update on sandbagging investigations
UK AISI's White Box Control team shares preliminary results on sandbagging investigations, using internal model inspection to detect models deliberately underperforming.
Why we're working on white box control
Introduces AISI's white box control research, which monitors or steers model internals to detect sandbagging, and reports preliminary progress on that detection.
1 item
How to evaluate control measures for LLM agents? A trajectory from today to superintelligence
Proposes a framework for scaling red-team control evaluations to agents' capability levels, defining five AI control levels with example rules and safety cases, and argues superintelligent agents may need new approaches.
1 item
How to evaluate control measures for AI agents?
UK AISI paper on AI control: how control protocols such as monitoring and human approval can constrain scheming LLM agents, and how to scale and evaluate those measures.
1 item
How we’re addressing the gap between AI capabilities and mitigations
UK AISI outlines its technical research agenda on safeguards against AI misuse and loss of control, covering the widening gap between capabilities and mitigations, plus hiring and grants.
1 item
A sketch of an AI control safety case
Sketches a "control safety case" argument that an internally deployed LLM agent cannot exfiltrate data, relying on red-team control evaluations and extrapolation claims.