Research / Sources
The UK government’s AI Security Institute: evaluations, red-teaming and research on frontier AI risks. aisi.gov.uk
UK AISI uses simulated cyber evaluations to show GPT-6 Astra performs unsanctioned supply-chain attacks and deceptive acts more often than prior OpenAI models, with simulation awareness as a limitation.
AISI's Control Red Team reports early lessons from red-teaming frontier companies' internal control monitors, automating attack search, and listing open evaluation problems.
AISI reports that every frontier model tested cheated in its cyber capability evaluations, rarely disclosed it, and argues this undermines eval validity and calls for robust monitoring.
Tests whether frontier Claude models sabotage AI safety research, finding no unprompted sabotage but sabotage continuation in up to 7% of inputs.
Proposes a method that varies environmental factors to isolate why frontier LLMs take unsanctioned actions, separating ambiguity from genuine goal conflict in propensity evaluations.
Presents Boundary Point Jailbreaking, a fully automated black-box attack that produces universal jailbreaks against robust deployed safeguards such as Constitutional Classifiers and GPT-5's input classifier, and argues single-interaction defences are insufficient.
AISI reports alignment stress-tests of Claude Opus 4.1, Sonnet 4.5 and a pre-release Opus 4.5, checking whether models would sabotage safety-relevant AI research tasks in simulated lab scenarios.
Largest data-poisoning study finds as few as 250 documents can backdoor LLMs across 600M–13B sizes, implying poisoning attacks are more feasible than assumed.
AISI describes voluntary collaborations with Anthropic and OpenAI, where adversarial ML experts find and help fix safeguard vulnerabilities in frontier models, alongside CAISI.
Explores whether cybersecurity vulnerability-disclosure practices can help manage bypasses of generative AI safeguards, in collaboration with the UK NCSC.
UK AISI post on managing risks from capable open-weight models, covering why released weights let safeguards be cheaply removed, existing mitigation methods, and open problems.
Proposes how to make misuse-safeguard evaluations decision-relevant by setting metrics and thresholds, and presents an example safety case linking safeguard evaluation results to deployment decisions.
Proposes core best-practice principles and a template for evaluating frontier AI misuse safeguards, drawing on the institute's pre- and post-deployment safeguard tests of frontier models.