Research / Sources
Policy research on advanced AI security and governance. www.iaps.ai
Argues that internal frontier AI systems face theft and sabotage risks plus internal rogue deployment, self-exfiltration and successor sabotage, and recommends security practices, expanded evaluations and government visibility.
A blog post on an Oxford Martin AI Governance Initiative paper examining which actors, such as governments, labs or third parties, should develop which AI model evaluations.
IAPS's response to a US BIS proposed rule recommends expanding reporting of frontier AI models and compute clusters to third-party evaluators, civil society, and public bodies, with ad hoc reporting and information sharing.
IAPS's response to a DoD RFI urges DoD-wide testing, evaluation and security standards for dual-use foundation models acquired through the defense industrial base, plus information-sharing with the US AI Safety Institute.
Analyzes 80 safety papers from Anthropic, Google DeepMind and OpenAI (2022-2024), mapping where corporate safety research is concentrated, where gaps lie, and how incentives shape it.
A UK policy memo recommending government market-shaping to grow the AI assurance technology industry, covering AI alignment, security and auditing tools, and the regulatory and demand uncertainties that hold it back.
A policy memo proposing Congress create an early warning system for novel AI-enabled threats, with BIS as a clearinghouse for dual-use capability evaluation reports from frontier developers.
Proposes a Coordinated Disclosure of Dual-Use Capabilities process, with a government clearinghouse receiving reports of dangerous AI capabilities so defenders get early warning and time to respond.
An IAPS brief analyzing three US FY2025 budget allocations for AI, including Commerce, export-control and Energy cybersecurity funding, for their impact on responsible advanced AI development.
Proposes responsible reporting of safety-critical information from frontier AI developers to government, industry and civil society, with voluntary and regulatory mechanisms for implementing it.
A systematic search of US federal agencies identifies regulatory case studies, using measures like expertise, risk assessment and enforcement, that could inform how advanced AI should be regulated.
Evaluates Anthropic's Responsible Scaling Policy against UK DSIT guidance on responsible capability scaling, arguing its risk thresholds should be lower and grounded in societal risk tolerances from other industries.
IAPS's response to NIST recommends that external actors get access and resources for scrutiny of dual-use foundation models, with incident response plans for dangerous capabilities and defense-in-depth.
Examines the US Federal Select Agent Program as a biosecurity precedent and draws lessons for AI regulation, covering development-phase licensing, regulatory expertise, and risk-based versus list-based approaches.
Argues that external scrutiny, including third-party auditing, red-teaming and researcher access, is needed for public accountability over frontier LLM decisions, and sets out six requirements under the ASPIRE framework with policy recommendations.
Proposes adapting cybersecurity frameworks (functional, lifecycle, and threat-based approaches) to help frontier AI developers and policymakers identify gaps in managing AI risks.
Argues that open-sourcing some highly capable foundation models may pose extreme risks outweighing benefits, and surveys alternative model-sharing strategies with policy recommendations for developers, standards bodies and governments.
Proposes an incident response framework and toolkit of deployment corrections that frontier AI developers can use to handle dangerous capabilities or behaviours discovered after model deployment, with recommendations for standardisation.