Research / Sources
The UK government’s AI Security Institute: evaluations, red-teaming and research on frontier AI risks. aisi.gov.uk
Infusion uses influence-function approximations to make tiny edits to training data that steer model behaviour, transferring across architectures and showing effects mainly on already-learned behaviours.
Largest data-poisoning study finds as few as 250 documents can backdoor LLMs across 600M–13B sizes, implying poisoning attacks are more feasible than assumed.
Shows that backdoor poisoning of LLM pretraining and fine-tuning needs a near-constant ~250 poisoned documents across model sizes from 600M to 13B, so larger models are not harder to poison.
Proposes inoculation prompting, prepending an instruction that elicits an unwanted trait during finetuning, which reduces that trait at test time and mitigates emergent misalignment, backdoors and subliminal learning.
NIST report giving a taxonomy and common terminology for adversarial machine learning attacks (evasion, poisoning, privacy, generative AI) and their mitigations across the AI lifecycle.