Research / Sources
Policy research on advanced AI security and governance. www.iaps.ai
Argues US policy should harden cyber defenses against autonomous offensive AI agents and proposes federal pilots of AI-enabled penetration testing and red teaming to address the threat.
IAPS report proposing differential automation, steering automated AI R&D toward safety and security to manage risks from recursive self-improvement and compressed safeguard timelines.
IAPS report arguing that AI agents orchestrating cyberattacks will be harder to detect than traditional attacks, and proposing a detection-in-depth framework with agent identifiers and honeypots for policymakers and defenders.
A policy memo proposing US executive actions to address national security risks from frontier AI, including securing model weights, automated cyber defense, information sharing, and government capacity.
A policy memo arguing that AI decision-support systems in military targeting fall outside DoD lethal-autonomy rules, and recommending safeguards for reliability and meaningful human oversight.
Report defines highly autonomous cyber-capable agents (HACCAs), forecasts their arrival, outlines five operational tactics for sustaining attacks, and analyzes strategic and tail risks of frontier AI offensive cyber capabilities.
Analyses the India AI Impact Summit 2026, covering its focus on demonstrable impact, US-China divergence on global AI governance, and competing visions of AI sovereignty.
A policy report arguing that government frontier AI systems need integrity protections against backdoors and secret loyalties, citing real attacks such as DeepSeek R1 poisoning.}
IAPS report mapping nine strategic visions for navigating powerful AI, assessing how each addresses misaligned AI takeover, great-power war, and AI-enabled power concentration, and which policy objectives are robust across them.
A IAPS memo analyses the MAIM deterrence argument, in which great powers would sabotage rivals' ASI races, reformulating its claims and assessing the evidence for and against each.
A policy memo analysing Anthropic's report of an AI agent autonomously running a cyber espionage campaign, and its national security implications for offensive cyber operations.
Report arguing the US government lacks AI security talent and fast hiring mechanisms for AI-related national security crises, outlining plausible scenarios and a typology of needed expertise.
Argues that compute should be treated as a strategic resource like oil or steel, scarce, controllable and concentrated, with implications for AI governance and competition.
IAPS experts review the US AI Action Plan through a national security and AGI-risk lens, praising its goals and identifying gaps and implementation considerations.
IAPS's response to a US RFI argues that AI reliability, security and safety R&D, plus an assurance consortium, are needed for trustworthy AI use in frontier science.
IAPS's response to a US federal RFI recommends AI R&D priorities on model security, reliability and capability assessments, supply chain security, and monitoring adversary AI, citing misalignment risks.
A survey of 53 experts ranking 105 AI reliability and security research areas by importance and tractability, finding dangerous-capability evals and multi-agent risks most promising.
A forecasting workshop with professional forecasters and US AI policy experts estimates the chance that a US government-led project would build and acquire the first advanced AI, and how it would be controlled.
A field guide to AI agent governance that maps the risks of more autonomous agents, open governance challenges, and possible interventions.
IAPS's comment on the US AI Action Plan argues for building trust in American AI through security and reliability standards, securing supply chains, denying adversaries advanced compute, and adaptive frameworks.
An IAPS linkpost arguing that AI companies' safety research leaves important gaps and that governments and philanthropists should fund work to fill them.
Analyzes 80 safety papers from Anthropic, Google DeepMind and OpenAI (2022-2024), mapping where corporate safety research is concentrated, where gaps lie, and how incentives shape it.
An IAPS brief analyzing three US FY2025 budget allocations for AI, including Commerce, export-control and Energy cybersecurity funding, for their impact on responsible advanced AI development.
A primer on Chinese AI chip manufacturing, covering the semiconductor industry and US export controls, to help forecast China's indigenous AI compute progress under compute governance.
Argues that public-interest institutions should fund alignment work reducing accident and misuse risks, identifies four ways alignment efforts could backfire, and proposes mitigations.
A consensus paper from many researchers outlining risks from advanced AI, including loss of human control, and proposing urgent priorities for AI R&D and governance.