Policy research on advanced AI security and governance. www.iaps.ai
Argues that contract terms will determine whether embedded third-party evaluators at frontier AI companies provide a real safety check, and calls for credible agreements now.
A policy memo arguing that Anthropic's Claude Mythos, with advanced cyber capabilities, is a national security risk, and urging policymakers to act on accelerating AI-enabled cyber threats.
Report defines highly autonomous cyber-capable agents (HACCAs), forecasts their arrival, outlines five operational tactics for sustaining attacks, and analyzes strategic and tail risks of frontier AI offensive cyber capabilities.
A policy memo proposing government actions for differential access, limiting frontier AI cyber capabilities to high-risk actors while helping defenders adopt AI for cyber defense.
A policy memo proposing a national CAISI+ center in the US Commerce Department for frontier model evaluations of catastrophic risks, emergency assessment, and AI security research, with budget and infrastructure recommendations.
IAPS's response to a US federal RFI recommends AI R&D priorities on model security, reliability and capability assessments, supply chain security, and monitoring adversary AI, citing misalignment risks.
A survey of 53 experts ranking 105 AI reliability and security research areas by importance and tractability, finding dangerous-capability evals and multi-agent risks most promising.
A blog post on an Oxford Martin AI Governance Initiative paper examining which actors, such as governments, labs or third parties, should develop which AI model evaluations.
A governance primer analysing the first wave of AI Safety Institutes (UK, US, Japan), comparing their shared characteristics, core functions of research, standards and cooperation, and challenges.
A policy memo proposing Congress create an early warning system for novel AI-enabled threats, with BIS as a clearinghouse for dual-use capability evaluation reports from frontier developers.
Proposes a Coordinated Disclosure of Dual-Use Capabilities process, with a government clearinghouse receiving reports of dangerous AI capabilities so defenders get early warning and time to respond.
IAPS's response to NIST recommends that external actors get access and resources for scrutiny of dual-use foundation models, with incident response plans for dangerous capabilities and defense-in-depth.
A consensus paper from many researchers outlining risks from advanced AI, including loss of human control, and proposing urgent priorities for AI R&D and governance.