Research / Sources
The UK government’s AI Security Institute: evaluations, red-teaming and research on frontier AI risks. aisi.gov.uk
Presents Boundary Point Jailbreaking, a fully automated black-box attack that produces universal jailbreaks against robust deployed safeguards such as Constitutional Classifiers and GPT-5's input classifier, and argues single-interaction defences are insufficient.
Introduces Boundary Point Jailbreaking, a fully black-box automated attack using only a single flagged/not-flagged bit, achieving universal jailbreaks against Constitutional Classifiers and GPT-5's input classifier.
UK AISI previews NeurIPS 2025 papers on malicious fine-tuning limits, data filtering as a defense, and a review of 445 LLM benchmarks finding popular ones often fail to measure what they claim.
AISI describes voluntary collaborations with Anthropic and OpenAI, where adversarial ML experts find and help fix safeguard vulnerabilities in frontier models, alongside CAISI.
Explores whether cybersecurity vulnerability-disclosure practices can help manage bypasses of generative AI safeguards, in collaboration with the UK NCSC.
UK AISI post on managing risks from capable open-weight models, covering why released weights let safeguards be cheaply removed, existing mitigation methods, and open problems.
Builds a few-shot input/output classifier defense that beats ShieldGemma, then shows a staged black-box attack reaching 71% ASR on ClearHarm against that safeguard pipeline.
NIST report giving a taxonomy and common terminology for adversarial machine learning attacks (evasion, poisoning, privacy, generative AI) and their mitigations across the AI lifecycle.
Shows that per-sample detection defenses for fine-tuning APIs are fundamentally limited, constructing benign-looking training data that covertly transmits harmful knowledge past monitors, tested on OpenAI's API.
Proposes core best-practice principles and a template for evaluating frontier AI misuse safeguards, drawing on the institute's pre- and post-deployment safeguard tests of frontier models.
Proposes five principles and a template for rigorous evaluation of frontier AI misuse safeguards, aiming to standardise how developers assess safeguards against harmful elicitation.
Joint UK and US AISI pre-deployment evaluation of upgraded Claude 3.5 Sonnet across biological, cyber, AI-development and safeguard domains, comparing against reference models.
UK AISI reports evaluations of five public LLMs for cyber, chem/bio expert knowledge, short-horizon agent tasks and jailbreak vulnerability, finding high jailbreak susceptibility and limited autonomous planning.