Research preprints from the scientific community. arxiv.org
Proposes double-blind AI safety evals using secure enclaves so evaluators keep prompts secret and model owners keep weights private, piloted on a Gemini model with the AILuminate benchmark.
Argues African fintech AI regulation lacks pre-deployment evaluation requirements, tests six commercial LLMs with a Nigerian-focused kit that finds fraud-script generation under framing and false positives on legitimate bank messages, and proposes a regulatory framework.
A review of AI-enabled biosecurity threats, evidence of uplift, limits of alignment transfer to biological models, interpretability for auditing removed capabilities, and defense-in-depth governance.
Proposes a six-attribute threat actor taxonomy so pre-release misuse evaluations of open-weight frontier models state adversary assumptions explicitly, comparably, and before testing.
Proposes harmonized capability thresholds for frontier AI misuse risks (cyber, biological) and automated AI R&D, so third parties can verify crossings and safety standards don't race to the bottom.
Proposes a Threshold Exceedance Criteria framework for CBRN uplift studies of frontier models and applies it in a pre-release expert-reviewed study, finding material uplift limited to radiological tasks.
Lays out 16 open technical challenges for managing safety risks of frontier open-weight AI models across training data, training, evaluation, deployment and ecosystem monitoring.
Argues that biological evaluations of AI scientist agents are interpretation-sensitive and gives experience-based guidance on design, scoring and documentation choices so policymakers can read risk results correctly.
Meta's safety and preparedness report for Muse Spark evaluates catastrophic risks in chem-bio, cyber and loss of control, finds elevated pre-mitigation risk, and justifies release after safeguards.
Argues that offensive-capability measurements of frontier AI overstate harm and credit models with scaffold capabilities, and calls for system-level, harm-grounded assessment and clearer accountability for harness builders.
Meta's preparedness report assesses the Code World Model for catastrophic-risk domains and misaligned propensities before open-weight release, concluding it adds no unusual frontier risk.
Builds a composite baseline of global cybercrime damages (about $500B annually) from surveys and spending data, to set capability thresholds for assessing frontier AI cyber risk.
The report defines highly autonomous cyber-capable AI agents, outlines their attack tactics and self-sustaining behaviour, flags loss-of-control and cyber-nuclear tail risks, and proposes seven policy recommendations.
Interviews with 16 practitioners identify methodological threats to validity in RCT-based human uplift studies of frontier AI, and map them to proposed solutions for governance decisions.
A position paper arguing that frontier AI safety should measure harmful capability uplift through human-centered evaluations, with methodological guidance and recommendations for developers, funders and regulators.
A technical report assessing frontier AI risks across cyber offense, persuasion, strategic deception, uncontrolled R&D, and self-replication, and proposing mitigation strategies for secure deployment.
Position paper arguing AI-agent-driven cyber attacks are inevitable, proposing offensive-capability benchmarks, trained vulnerability-discovery agents, and governance that confines offensive agents to audited cyber ranges with staged release.
Shows that capabilities once requiring frontier compute now run on consumer hardware, so harm campaigns are feasible at low cost, and argues compute-based AI governance leaves serious security gaps.
Reports an evaluation of Amazon's Nova 2.0 Lite under its Frontier Model Safety Framework across CBRN, offensive cyber and automated AI R&D risk domains using benchmarks, red-teaming and uplift studies.
Argues that compute thresholds misjudge AI weaponization risk, proposes an intent-and-capability definition grounded in international humanitarian law, and outlines live benchmarks across data, algorithms and compute.
Applies a quantitative risk model to AI-enabled cyber offense, mapping Cybench and BountyBench scores via Delphi and LLM-simulated experts to estimate attack uplift for risk management and policy.
Proposes a six-step methodology combining scenario building with quantitative risk estimation to model systemic AI risks like cyber offense and loss of control, validated on LLM-enabled cyber offense.
Pilot implementation of the Bacterial Biothreat Benchmark dataset for assessing biosecurity risk from frontier LLMs via model responses, human evaluation, and risk analysis to guide mitigation priorities.
Describes generation of the B3 bacterial biothreat benchmark dataset of 1,010 uplift-diagnostic prompts for assessing whether frontier LLMs could aid bioweapon development.
Compares the ChemBio safety evaluation reporting in o3, Claude 4 and Gemini 2.5 Pro model reports against the STREAM standard, finding gaps in transparency across all three.
An international report update assessing how frontier AI capabilities have advanced since 2024 and how these gains affect biological, cyber, and control risks, with updated risk assessments.
A policy memorandum recommending that US defence and intelligence agencies assess frontier AI misalignment and scheming through control and scheming evaluations to ensure reliability and governability.
Benchmarks frontier LLMs as agents in simulated multi-turn military conflicts, finding they violate distinction principles by targeting civilians and show inter-model differences in harm tolerance, proposing pre-deployment metrics.
Argues frontier multimodal models will automate military and strategic intelligence, reports a preliminary uplift study, proposes a taxonomy of intelligence questions, and gives policy recommendations for nation-states.
Proposes STREAM, a reporting standard for disclosing chemical and biological dangerous-capability evaluation results in AI model reports, developed with 23 experts and including a template.
Assesses frontier AI models across seven risk areas, including cyber, bio, scheming and self-replication, using red and yellow lines; finds no model crosses a red line.
Amazon evaluates its Nova Premier model against frontier safety thresholds in CBRN, offensive cyber, and automated AI R&D domains, using benchmarks, red-teaming, and uplift studies, concluding it is safe to release.
Fine-tunes an open-source LLM on partisan Reddit discussions and finds its comments are often indistinguishable from human ones, raising concerns about manipulation and disinformation.
Crowdsourced CTF competitions with open AI tracks show AI agents reach top placements and solve hour-or-less cyber challenges, proposing elicitation bounties to track offensive cyber capabilities.
Analyzes cyber risks from open-weight GPAI models such as DeepSeek-R1, critiques the EU AI Act and GPAI Code of Practice gaps, and proposes capability-focused evaluation and policy paths.
Analyses frontier AI's effect on cybersecurity through benchmarks, agent evaluations and an expert survey, finding attackers benefit more than defenders and calling for better benchmarks, secure agents and pre-deployment testing.
Proposes guidance for prioritizing offensive AI research, using SDGs and interpretability, and evaluates CTF-solving vulnerability agents and AI-powered malware for societal benefit versus risk.
Argues for deeper-than-black-box external evaluations of general-purpose AI models, covering the security threats of remote evaluation, technical safeguards, and governance recommendations.
Proposes principles and specific intolerable-risk thresholds for frontier AI across eight risk categories, including CBRN, cyber, model autonomy and deception, to guide policymakers and industry commitments.
Position paper arguing LLM cyber-risk evaluations miss real-world impact, proposing a risk assessment framework that adds threat-actor adoption and impact, and applying it to frontier cybersecurity assistants.