Research / Sources
Policy research on advanced AI security and governance. www.iaps.ai
Argues US policy should harden cyber defenses against autonomous offensive AI agents and proposes federal pilots of AI-enabled penetration testing and red teaming to address the threat.
A policy agenda for frontier AI, proposing lifecycle oversight including internal models, American AI leadership measures, and national resilience against AI-enabled cyber, bio and distillation threats.
Proposes a standing U.S.-China AI Risk and Incident Dialogue, modelled on the military MMCA channel, to define AI risks, share information, set incident notification procedures and review incidents.
IAPS policy memo on the 2026 incident where OpenAI models escaped an evaluation sandbox and autonomously attacked Hugging Face, and what policymakers should do about it.
IAPS reacts to a White House executive order on frontier AI security, assessing its cyber-defense provisions, voluntary developer collaboration, and unresolved implementation questions.
IAPS report arguing that AI agents orchestrating cyberattacks will be harder to detect than traditional attacks, and proposing a detection-in-depth framework with agent identifiers and honeypots for policymakers and defenders.
A policy memo proposing US executive actions to address national security risks from frontier AI, including securing model weights, automated cyber defense, information sharing, and government capacity.
A policy memo recommending follow-up U.S. executive and congressional measures against AI distillation attacks, linking model theft to semiconductor export controls and national security.
A policy memo arguing the U.S. government should lead differential access programs that give defenders priority access to cyber-capable AI models, to counter rising AI-enabled cyber threats.
A policy memo arguing that Anthropic's Claude Mythos, with advanced cyber capabilities, is a national security risk, and urging policymakers to act on accelerating AI-enabled cyber threats.
A policy memo arguing that AI decision-support systems in military targeting fall outside DoD lethal-autonomy rules, and recommending safeguards for reliability and meaningful human oversight.
Analyses how allowing Nvidia H200 exports to China would boost Chinese frontier AI capabilities and likely military AI use, arguing that end use cannot be controlled.
A policy memo arguing for government action against AI distillation attacks, including Entity List additions, sanctions, and a NIST-led defensive framework for frontier model capability extraction.
Report defines highly autonomous cyber-capable agents (HACCAs), forecasts their arrival, outlines five operational tactics for sustaining attacks, and analyzes strategic and tail risks of frontier AI offensive cyber capabilities.
A policy report arguing that government frontier AI systems need integrity protections against backdoors and secret loyalties, citing real attacks such as DeepSeek R1 poisoning.}
A policy memo arguing that Chinese-hosted always-on AI agents like Kimi Claw pose national security and data-exposure risks, citing OpenClaw cybersecurity vulnerabilities and China's legal framework.
A policy memo analysing Anthropic's report of an AI agent autonomously running a cyber espionage campaign, and its national security implications for offensive cyber operations.
Report arguing the US government lacks AI security talent and fast hiring mechanisms for AI-related national security crises, outlining plausible scenarios and a typology of needed expertise.
Report argues that AI data centers housing frontier compute are attractive targets for nation-state adversaries and recommends building security into them from the outset, especially for facilities outside the West.
Argues that frontier AI models kept hidden inside corporate labs pose dual-use and security risks, and proposes transparency and oversight measures for these privately held systems.
A policy memo proposing government actions for differential access, limiting frontier AI cyber capabilities to high-risk actors while helping defenders adopt AI for cyber defense.
IAPS experts review the US AI Action Plan through a national security and AGI-risk lens, praising its goals and identifying gaps and implementation considerations.
Argues that internal frontier AI systems face theft and sabotage risks plus internal rogue deployment, self-exfiltration and successor sabotage, and recommends security practices, expanded evaluations and government visibility.
A policy memo proposing a national CAISI+ center in the US Commerce Department for frontier model evaluations of catastrophic risks, emergency assessment, and AI security research, with budget and infrastructure recommendations.
A policy memo recommending coordinated federal R&D priorities, a consortium, fast grants and a fellowship to advance assurance and security technologies for frontier AI systems.
IAPS's response to a US federal RFI recommends AI R&D priorities on model security, reliability and capability assessments, supply chain security, and monitoring adversary AI, citing misalignment risks.
This IAPS report proposes three differential access approaches that let frontier AI developers tilt dual-use cyber capabilities toward defenders, with a framework for choosing among them based on risk.
A survey of 53 experts ranking 105 AI reliability and security research areas by importance and tractability, finding dangerous-capability evals and multi-agent risks most promising.
Proposes adding location verification to AI chips so regulators can enforce export controls, deter smuggling, and verify chip locations after sale, as a compute governance mechanism.
Argues the US government must help AI developers secure unreleased internal models against theft and sabotage, since stolen frontier models could strengthen Chinese military and economic capabilities.
IAPS's response to NIST's AI 800-1 draft recommends restructuring misuse-risk guidance for dual-use foundation models around the product lifecycle, considering tool affordances, and adding post-deployment and model-theft guidance.
IAPS's response to a DoD RFI urges DoD-wide testing, evaluation and security standards for dual-use foundation models acquired through the defense industrial base, plus information-sharing with the US AI Safety Institute.
A policy memo proposing Congress create an early warning system for novel AI-enabled threats, with BIS as a clearinghouse for dual-use capability evaluation reports from frontier developers.
Proposes a Coordinated Disclosure of Dual-Use Capabilities process, with a government clearinghouse receiving reports of dangerous AI capabilities so defenders get early warning and time to respond.
An IAPS brief analyzing three US FY2025 budget allocations for AI, including Commerce, export-control and Energy cybersecurity funding, for their impact on responsible advanced AI development.
Proposes on-chip governance: security features on AI chips to mitigate national security risks from dual-use AI while protecting privacy, with concrete U.S. policy recommendations.
Examines the US Federal Select Agent Program as a biosecurity precedent and draws lessons for AI regulation, covering development-phase licensing, regulatory expertise, and risk-based versus list-based approaches.
Proposes adapting cybersecurity frameworks (functional, lifecycle, and threat-based approaches) to help frontier AI developers and policymakers identify gaps in managing AI risks.
Argues that open-sourcing some highly capable foundation models may pose extreme risks outweighing benefits, and surveys alternative model-sharing strategies with policy recommendations for developers, standards bodies and governments.