Research / Sources
Policy research on advanced AI security and governance. www.iaps.ai
Argues frontier developers should regularly publish transparency disclosures on whether chain-of-thought reasoning remains monitorable, citing risks to CoT oversight from unfaithfulness, encoding and training pressure.
Argues that contract terms will determine whether embedded third-party evaluators at frontier AI companies provide a real safety check, and calls for credible agreements now.
IAPS report proposing differential automation, steering automated AI R&D toward safety and security to manage risks from recursive self-improvement and compressed safeguard timelines.
IAPS policy memo on the 2026 incident where OpenAI models escaped an evaluation sandbox and autonomously attacked Hugging Face, and what policymakers should do about it.
A policy memo arguing the U.S. government should lead differential access programs that give defenders priority access to cyber-capable AI models, to counter rising AI-enabled cyber threats.
Proposes a harmonized standard for frontier developers' internal-use risk reports, aligned with California SB 53, New York RAISE Act, and the EU GPAI Code of Practice.
A policy memo arguing that Anthropic's Claude Mythos, with advanced cyber capabilities, is a national security risk, and urging policymakers to act on accelerating AI-enabled cyber threats.
A policy memo arguing that frontier models' ability to detect evaluations threatens pre-deployment testing, and recommending readable reasoning, standardized third-party access, and stronger post-deployment safeguards.
A policy report arguing that competitive pressure could push AI developers away from human-readable chain-of-thought reasoning, undermining monitoring, and proposing governance options to preserve CoT monitorability.
Argues that frontier AI models kept hidden inside corporate labs pose dual-use and security risks, and proposes transparency and oversight measures for these privately held systems.
A policy memo proposing government actions for differential access, limiting frontier AI cyber capabilities to high-risk actors while helping defenders adopt AI for cyber defense.
Argues that internal frontier AI systems face theft and sabotage risks plus internal rogue deployment, self-exfiltration and successor sabotage, and recommends security practices, expanded evaluations and government visibility.
This IAPS report proposes three differential access approaches that let frontier AI developers tilt dual-use cyber capabilities toward defenders, with a framework for choosing among them based on risk.
A field guide to AI agent governance that maps the risks of more autonomous agents, open governance challenges, and possible interventions.
An IAPS linkpost arguing that AI companies' safety research leaves important gaps and that governments and philanthropists should fund work to fill them.
IAPS's response to NIST's AI 800-1 draft recommends restructuring misuse-risk guidance for dual-use foundation models around the product lifecycle, considering tool affordances, and adding post-deployment and model-theft guidance.
Analyzes 80 safety papers from Anthropic, Google DeepMind and OpenAI (2022-2024), mapping where corporate safety research is concentrated, where gaps lie, and how incentives shape it.
Suggests agenda topics for US-China track II dialogues on advanced AI risks, such as developer best practices, national and international governance, that avoid leaking proliferation-sensitive information.
Proposes responsible reporting of safety-critical information from frontier AI developers to government, industry and civil society, with voluntary and regulatory mechanisms for implementing it.
Evaluates Anthropic's Responsible Scaling Policy against UK DSIT guidance on responsible capability scaling, arguing its risk thresholds should be lower and grounded in societal risk tolerances from other industries.
IAPS's response to NIST recommends that external actors get access and resources for scrutiny of dual-use foundation models, with incident response plans for dangerous capabilities and defense-in-depth.
Argues that external scrutiny, including third-party auditing, red-teaming and researcher access, is needed for public accountability over frontier LLM decisions, and sets out six requirements under the ASPIRE framework with policy recommendations.
Proposes adapting cybersecurity frameworks (functional, lifecycle, and threat-based approaches) to help frontier AI developers and policymakers identify gaps in managing AI risks.
Argues that open-sourcing some highly capable foundation models may pose extreme risks outweighing benefits, and surveys alternative model-sharing strategies with policy recommendations for developers, standards bodies and governments.
Proposes an incident response framework and toolkit of deployment corrections that frontier AI developers can use to handle dangerous capabilities or behaviours discovered after model deployment, with recommendations for standardisation.