Research / Sources
Research preprints from the scientific community. arxiv.org
Proposes double-blind AI safety evals using secure enclaves so evaluators keep prompts secret and model owners keep weights private, piloted on a Gemini model with the AILuminate benchmark.
Uses Project Glasswing's restricted release of a vulnerability-finding model to show access helped firms find and fix more flaws, without evident attacker gains, but created cross-country asymmetry.
Proposes Agentic Process Evidence, a signed, tamper-evident attestation of AI coding agent sessions linked to outcomes, enabling policy-as-code governance and accountability in the software development lifecycle.
Analyses 139 CivitAI guides to show how general-purpose image-generation workflows, mostly built on open-weight models, can be configured into non-consensual intimate imagery, and argues safeguards should target identity generation and align with EU regulation.
Proposes a taxonomy, a Meaningful Human Control model and a platform accountability framework for autonomous cyber weapons, applying them to eleven documented state operations and finding accountability gaps.
Proposes a multi-tier access framework for frontier AI that ties who can obtain capabilities to risk, arguing open-weight release forfeits safety levers and safety fine-tuning can be stripped cheaply.
Introduces on-policy attacks that steer LLM agents' product choices by planting fabricated web content linked to a truthful negation, showing frontier agents can be manipulated without violating platform rules.
Proposes a computational reflective equilibrium scaffold for LLM agents that encodes principles, governance, and provenance, with a prompt-injection quarantine theorem and two case studies.
Develops a non-operational failure-mode framework for how advanced AI could cause extinction or permanent disempowerment via misalignment, human misuse, organizational failure and competitive deployment, without requiring intent.
Surveys LLM applications in mobility and finds security, privacy and reliability are largely neglected, with minimal AI Act compliance such as vulnerability assessment and risk management.
Argues African fintech AI regulation lacks pre-deployment evaluation requirements, tests six commercial LLMs with a Nigerian-focused kit that finds fraud-script generation under framing and false positives on legitimate bank messages, and proposes a regulatory framework.
A review of trustworthiness risks in LLM agents, covering prompt injection, memory contamination and goal misgeneralisation, plus a proposed six-phase lifecycle framework for building secure agentic systems.
A review of AI-enabled biosecurity threats, evidence of uplift, limits of alignment transfer to biological models, interpretability for auditing removed capabilities, and defense-in-depth governance.
A review of authorization architectures for tool-using AI agents, proposing a principal hierarchy, seven structural requirements, and a reference architecture, and identifying runtime enforcement as a key gap.
A survey of security and trustworthiness for agentic AI, covering threat surfaces, a zero-trust defense-in-depth architecture, evaluation benchmarks, and mapping of technical controls to AI governance frameworks.
Proposes a unified policy architecture for governing autonomous AI agents, tools, memory and agent-to-agent interactions in enterprise systems, covering authorisation, runtime obligations, human approvals and audit.
Profiles the ecosystem of uncensored open-weight LLMs on HuggingFace and GitHub, finding widespread redistribution that persists after upstream removal and many malicious downstream applications.
Argues that LLM safeguard metrics like refusal and attack success rates don't establish deployment-level safety, and that system-level evidence is needed to judge whether a safeguard makes a deployment safer.
Argues non-great-power conflict could be a major catastrophic risk channel for advanced AI, testing pathways via war escalation, terrorism and AI loss of control, and identifies shared intermediate variables.
Proposes a six-attribute threat actor taxonomy so pre-release misuse evaluations of open-weight frontier models state adversary assumptions explicitly, comparably, and before testing.
Proposes a framework linking STPA hazard analysis, component testing and system modelling to trace LLM trading manipulation into financial-system harms, showing adversarial inputs can increase bank failures and contagion.
Examines how US export controls on frontier AI access interact with cyber threats, arguing access can be revoked and proposing layered strategies for small and middle powers' AI sovereignty.
Proposes a 32-stage security-oriented lifecycle model for LLM systems and maps it against NIST AI RMF, EU AI Act and ISO/IEC 42001, finding governance evidence concentrates at deployment-facing stages.
A systematization of knowledge that proposes a five-dimensional framework for how frontier AI reshapes security risk in critical infrastructure, and sets a system-level assurance research agenda.
Proposes harmonized capability thresholds for frontier AI misuse risks (cyber, biological) and automated AI R&D, so third parties can verify crossings and safety standards don't race to the bottom.
Proposes a Threshold Exceedance Criteria framework for CBRN uplift studies of frontier models and applies it in a pre-release expert-reviewed study, finding material uplift limited to radiological tasks.
Proposes GRAM, a pretraining method using gradient-routed modules so dual-use capabilities can be removed at inference, offering cheaper access control than data filtering and resisting finetuning recovery better than unlearning.
Large-scale study measuring jailbreak-driven unsafe image generation across 200+ open Hugging Face text-to-image models, introducing a refined success metric and identifying high-risk releases.
Proposes LARA, a Lagrangian dual-based reward augmentation for inference-time alignment of frozen LLMs under safety cost constraints, improving the helpfulness-harmlessness tradeoff with Best-of-N.
Lays out 16 open technical challenges for managing safety risks of frontier open-weight AI models across training data, training, evaluation, deployment and ecosystem monitoring.
Argues that autonomous AI-mediated cyber operations strain IHL's direct-causation test and that existing AI governance instruments fail to log goal-specification granularity, the property the paper identifies as central.
Practitioner paper mapping LLM-agent threats such as prompt injection and tool abuse onto US and EU financial regulation, with production KYC architecture patterns and negative results.
A field-wide prioritized agenda for AI security, drawn from expert interviews and a workshop, covering frontier system protection, policy, technical assurance, and governing agentic AI under adversarial pressure.
Argues that frontier AI models have become decisive cyber-offense tools, that access to them is gated by a small group of developers, and that Africa must respond through threat-intelligence sharing and governance within a six-to-twelve-month window.
Argues that biological evaluations of AI scientist agents are interpretation-sensitive and gives experience-based guidance on design, scoring and documentation choices so policymakers can read risk results correctly.
Position paper arguing that autonomous agents should be engineered to refuse user requests responsibly, with justifications, override pathways, security risk tracking, and liability transfer.
Position paper arguing AI researchers must help lead arms control research, using nuclear-deterrence lessons, to reduce risks from frontier AI models integrated into military systems.
Position paper arguing that preventing AI-generated child sexual abuse material requires new AI safety approaches, outlining 15 open problems across the development lifecycle and recommendations for researchers, developers and policymakers.
Position paper arguing that alignment techniques are dual-use and could be misused by malicious actors for censorship and manipulation, and proposing mitigation strategies.
This policy memo argues that misaligned AI models with privileged access should be treated as insider risks in high-stakes government contexts, and recommends adapting continuous evaluation and monitoring to them.