Research / Sources
Policy research on advanced AI security and governance. www.iaps.ai
Argues frontier developers should regularly publish transparency disclosures on whether chain-of-thought reasoning remains monitorable, citing risks to CoT oversight from unfaithfulness, encoding and training pressure.
A policy memo proposing how the US and China should operationalize their agreed AI incident notification channel, recommending a text-based template, who to notify, and when to activate it.
Argues that contract terms will determine whether embedded third-party evaluators at frontier AI companies provide a real safety check, and calls for credible agreements now.
Proposes near-term verification mechanisms, covering end-location, end-user and end-use checks, to help US policymakers enforce AI chip export controls within about a year.
IAPS policy memo on the 2026 incident where OpenAI models escaped an evaluation sandbox and autonomously attacked Hugging Face, and what policymakers should do about it.
IAPS reacts to a White House executive order on frontier AI security, assessing its cyber-defense provisions, voluntary developer collaboration, and unresolved implementation questions.
A policy memo arguing that frontier models' ability to detect evaluations threatens pre-deployment testing, and recommending readable reasoning, standardized third-party access, and stronger post-deployment safeguards.
Proposes BIS-accredited, for-profit export auditors that inspect shipments to detect AI chip diversion, arguing this would scale US export enforcement beyond the agency's own capacity.
Analyses how international AI agreements could be verified, finding data-center regulation verifiable within years using narrow hardware and privacy-preserving computation, while mobile weapons regulation faces severe barriers.
Argues that internal frontier AI systems face theft and sabotage risks plus internal rogue deployment, self-exfiltration and successor sabotage, and recommends security practices, expanded evaluations and government visibility.
IAPS's response to a US RFI argues that AI reliability, security and safety R&D, plus an assurance consortium, are needed for trustworthy AI use in frontier science.
A policy memo proposing a national CAISI+ center in the US Commerce Department for frontier model evaluations of catastrophic risks, emergency assessment, and AI security research, with budget and infrastructure recommendations.
A policy memo recommending coordinated federal R&D priorities, a consortium, fast grants and a fellowship to advance assurance and security technologies for frontier AI systems.
IAPS's response to a US federal RFI recommends AI R&D priorities on model security, reliability and capability assessments, supply chain security, and monitoring adversary AI, citing misalignment risks.
A blog post on an Oxford Martin AI Governance Initiative paper examining which actors, such as governments, labs or third parties, should develop which AI model evaluations.
Commentary on how the International Network of AI Safety Institutes should set priorities, structure membership and coordinate on safety standards, information sharing and evaluations ahead of the November 2024 San Francisco convening.
Identifies Chinese government-linked institutions that could serve as counterparts to the US and UK AI Safety Institutes, based on a systematic review of open sources.
IAPS's response to a US BIS proposed rule recommends expanding reporting of frontier AI models and compute clusters to third-party evaluators, civil society, and public bodies, with ad hoc reporting and information sharing.
A governance primer analysing the first wave of AI Safety Institutes (UK, US, Japan), comparing their shared characteristics, core functions of research, standards and cooperation, and challenges.
IAPS's response to NIST's AI 800-1 draft recommends restructuring misuse-risk guidance for dual-use foundation models around the product lifecycle, considering tool affordances, and adding post-deployment and model-theft guidance.
IAPS's response to a DoD RFI urges DoD-wide testing, evaluation and security standards for dual-use foundation models acquired through the defense industrial base, plus information-sharing with the US AI Safety Institute.
Argues that international scientific assessments of AI risks need a coordinated process among the UK-led report, the UN, the OECD and AI Safety Institutes, and discusses how these actors could work together.
A UK policy memo recommending government market-shaping to grow the AI assurance technology industry, covering AI alignment, security and auditing tools, and the regulatory and demand uncertainties that hold it back.
A policy memo proposing Congress create an early warning system for novel AI-enabled threats, with BIS as a clearinghouse for dual-use capability evaluation reports from frontier developers.
Proposes a Coordinated Disclosure of Dual-Use Capabilities process, with a government clearinghouse receiving reports of dangerous AI capabilities so defenders get early warning and time to respond.
Evaluates Anthropic's Responsible Scaling Policy against UK DSIT guidance on responsible capability scaling, arguing its risk thresholds should be lower and grounded in societal risk tolerances from other industries.
IAPS's response to NIST recommends that external actors get access and resources for scrutiny of dual-use foundation models, with incident response plans for dangerous capabilities and defense-in-depth.
Argues that external scrutiny, including third-party auditing, red-teaming and researcher access, is needed for public accountability over frontier LLM decisions, and sets out six requirements under the ASPIRE framework with policy recommendations.
Proposes an incident response framework and toolkit of deployment corrections that frontier AI developers can use to handle dangerous capabilities or behaviours discovered after model deployment, with recommendations for standardisation.