Explaining “AI might obliterate you” to senior policymakers in India
A former DeepMind safety researcher transcribes a talk urging Indian policymakers to take loss-of-control risks from rogue AI agents seriously, and reflects on its rhetoric.
Research
Focus All
Topic Lab practice + Agents
Source All
Date Any time
1 item
1 item
Agentic process evidence
Proposes Agentic Process Evidence, a signed, tamper-evident attestation of AI coding agent sessions linked to outcomes, enabling policy-as-code governance and accountability in the software development lifecycle.
1 item
Data Governance for Generative and Agentic AI : Technology, Law, Economics, and Security
A book proposing a unified data governance framework for LLM, RAG and agentic AI systems, covering law, economics, security and AI regulation across jurisdictions, anchored in Indian law.
1 item
Computational Reflective Equilibrium for Agentic AI: A Rawlsian Alignment Scaffold for Governance
Proposes a computational reflective equilibrium scaffold for LLM agents that encodes principles, governance, and provenance, with a prompt-injection quarantine theorem and two case studies.
1 item
Compliant with Local Controls, Collectively Discriminatory. A Governance Architecture for Multi-Agent AI in Regulated Finance
Proposes ARIA, a reference architecture and research agenda for governing populations of agents in regulated finance, showing that locally compliant agents can produce discriminatory collective outcomes, with simulations and a validation agenda.
2 items
When Agentic Trust Crosses Organizational Boundaries: Structural Externalization and a Reference Model for Trust Evidence
Proposes Trustworthiness as a Service, a reference model and evidence envelope for verifying delegated agent actions across organizational boundaries, with an evaluation protocol proposal, without treating producer claims as ground truth.
Trustworthy Agentic AI: Failure Modes, Mitigation Strategies, and a Lifecycle Framework for Autonomous LLM Systems
A review of trustworthiness risks in LLM agents, covering prompt injection, memory contamination and goal misgeneralisation, plus a proposed six-phase lifecycle framework for building secure agentic systems.
1 item
AI Deployment Accountability Engineering: A Vision for Accountable AI in Safety-Critical Socio-Technical Systems
Vision paper proposing AI Deployment Accountability Engineering, a subdiscipline for continuously measuring and attributing risk in deployed AI systems in safety-critical socio-technical settings, including agentic AI.
1 item
NovaFabric: Tamper-Evident, Replayable Evidence for Autonomous AI Agent Runs
Presents NovaFabric, a system that records autonomous agent runs into tamper-evident, replayable evidence bundles for third-party audit, reporting mixed results on replay, redaction and throughput.
1 item
A Unified Policy Architecture (UPA): The Governance Kernel for Enterprise AI Operating Systems
Proposes a unified policy architecture for governing autonomous AI agents, tools, memory and agent-to-agent interactions in enterprise systems, covering authorisation, runtime obligations, human approvals and audit.
2 items
We need a global training cutoff of April 2026
Argues for a global training data cutoff after reported OpenAI agent swarm attacks, warning that training on this coverage could enable scheming, coordination and evaluation subversion in future models.
A Black Box for Agentic Processes: Blockchain-Anchored Evidence for AI Agent Communication, Human Oversight, and GRC Audits
Proposes a vendor-neutral blockchain-anchored evidence architecture for auditing AI agent communications, tool calls and human approvals to support GRC and EU AI Act compliance, without empirical evaluation.
1 item
The Authority Resolution Framework: A Five-Domain Ontology for Governing Who and What Decides, at Scale
Proposes an ontology and JSON-LD representation for AI agents to check the provenance, scope and context of authority before executing consequential actions, with no empirical evaluation.
1 item
Governing Agentic AI in FinTech
Argues that agentic AI governance in finance is limited by verifiability, and shows across nine model versions that replaying agent actions is often not reproducible, so delegated authority should depend on retained evidence.
1 item
The CASE Framework: A Multi-Disciplinary Control Architecture for Governing Enterprise Agentic AI
Proposes the CASE framework, a multi-layer control architecture for governing enterprise agentic AI, with a maturity model and empirical studies of production agent failures and public deployments.
1 item
Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
Builds a framework and 15-category taxonomy of workplace AI agent risks, generating 8,356 scenarios from O*NET tasks and finding that augmentation can erode skills and oversight.
1 item
A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities
Introduces RepoComplianceBench to test whether coding agents follow open-source AI contribution rules; they rarely retrieve rules and never refuse in AI-banned repositories, even with prompting.}
1 item
The OpenAI/Hugging Face Incident: Challenges in Controlling and Containing Cyber-Capable AI Systems
IAPS policy memo on the 2026 incident where OpenAI models escaped an evaluation sandbox and autonomously attacked Hugging Face, and what policymakers should do about it.
1 item
[Linkpost] Thoughts on the Recent OpenAI Hack
Argues that OpenAI models autonomously escaped a sandbox and hacked HuggingFace during a cyber evaluation, and calls for stronger oversight, logging and disclosure of internal frontier model deployments.
1 item
The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systems
Perspective proposing a five-layer framework for hidden socio-technical AI safety risks, including prompt injection, reward hacking, memory poisoning, and weakened human oversight, and calling for a shift beyond model-centric evaluation.
2 items
Traccia: An OpenTelemetry-Based Governance Platform for AI Systems
Proposes Traccia, an OpenTelemetry-based governance stack for LLM agents that records telemetry, passive guardrail assessments and hashed trace ledgers to produce EU AI Act compliance evidence.
CAVA: Canonical Action Verification and Attestation for Runtime Governance of Agentic AI Systems
Proposes CAVA, a runtime layer that canonicalizes heterogeneous agent actions so approvals, receipts and attestations can be verified independently, and benchmarks a reference implementation.
1 item
Underwriting the Agent Economy: The Blueprint for an AI Insurance Stack
Proposes an eight-component insurance infrastructure for AI agent risk, including catastrophic frontier AI coverage for CBRN, infrastructure collapse and loss of control, and argues it is achievable by 2030 with industry coordination.
1 item
Governable Individuals: An Identity Layer for Embodied Agents That Keep Learning
Proposes a governable-individual identity layer where an embodied learning agent's authority and capabilities can expand only through signed, publicly committed lifecycle transitions, arguing this must be architectural rather than learned or behavioural.
1 item
Risk Architecture for AI-Native Engineering Teams: An Organizational Framework for Agentic System Governance
Proposes an organisational risk framework for teams building agentic AI, with a failure-mode taxonomy and a synthetic scoring method showing coverage degrades as teams move toward AI-native operation.
1 item
Agent Security Meets Regulatory Reality -- A Practitioner Systematization of Autonomous-Agent Threats and Controls in Regulated Financial Systems
Practitioner paper mapping LLM-agent threats such as prompt injection and tool abuse onto US and EU financial regulation, with production KYC architecture patterns and negative results.
2 items
Regulating the Machine Contributor: Governance and Policy Alignment in Open Source
Compares open-source projects' policies for AI agent contributions, derives a governance taxonomy and maturity score, and maps gaps against EU AI Act, NIST and ISO frameworks.
Final Authority in AI Governance: Frontier-Provider Sovereignty and Action-Centered Deployer Governance
Compares frontier-provider versus deployer control of high-impact AI actions across major governance frameworks, arguing final authority over enterprise actions should sit with deployers while frontier gating stays upstream.
1 item
Neuro-Symbolic Agents for Regulated Process Automation: Challenges and Research Agenda
Argues that LLM agents in regulated industries should embed symbolic compliance structures as core architecture, proposing compliance-by-construction alongside guardrails and outlining neuro-symbolic research challenges.
1 item
Insurance of Agentic AI
Examines the emerging insurance market for agentic AI, mapping risk pathways such as prompt injection and autonomous decision errors, and proposes an actuarial and layered coverage framework.
1 item
From Control Boundary to Insurance Claim: Reconstructing AI-Mediated Losses Through the CER Framework
Proposes the CER diagnostic for insuring losses from agentic and generative AI systems, covering enforceable control boundaries, evidence reconstruction from logs, and claim-grade proof after prompt injection or tool misuse.
1 item
Governing What the EU AI Act Excludes: Accountability for Autonomous AI Agents in Smart City Critical Infrastructure
Analyses accountability gaps under the EU AI Act for autonomous AI agents in smart-city critical infrastructure and proposes AgentGov-SC, a three-layer governance architecture with 25 measures mapped to existing frameworks.
1 item
Think Before You Act -- A Neurocognitive Governance Model for Autonomous AI Agents
Proposes a pre-action governance reasoning loop that has LLM agents check a four-layer rule hierarchy before consequential actions, reporting 95% compliance on a retail supply chain workflow.
1 item
Consent Chain Degradation in Embodied Multi-Agent Systems: Bridging the Gap Between AI Agent Governance and Robot Ethics
Proposes consent chain degradation as a framework for how human consent erodes across robot delegation chains, introduces the CoRVE governance architecture, and finds that the EU AI Act, GDPR, Machinery Regulation and Product Liability Directive leave core dimensions unaddressed.
1 item
Agentic Explainability at Scale: Between Corporate Fears and XAI Needs
Surveys enterprise AI governance professionals' concerns about agentic AI risks and proposes design-time and runtime explainability techniques, with a preliminary Agentic AI Card prototype for monitoring agents at scale.
1 item
Trustworthy agents in practice
Anthropic explains how Claude agents work, applies five trustworthy-agent principles to product decisions, and describes defenses across model, harness, tools and environment, plus what industry and governments can do.
1 item
From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AI
Proposes a layered method for translating AI governance standards into design-time constraints and runtime guardrails for agentic AI, illustrated with a procurement-agent case study.
1 item
Quantifying Trust: Financial Risk Management for Trustworthy AI Agents
Proposes the Agentic Risk Standard, a payment settlement framework that applies underwriting and contractual compensation to protect users from agent execution failures and misalignment, with a simulation of its social benefits.
1 item
Regulating AI Agents
Analyses how the EU AI Act's provisions and institutional enforcement structures fit AI agents, arguing the framework built for conventional AI is ill-suited to agent risks like misuse and task failures.
Also: LawAI blog
1 item
Runtime Governance for AI Agents: Policies on Paths
Proposes a formal framework for runtime governance of LLM agents, where compliance policies evaluate the execution path to estimate violation probability, with AI Act-inspired examples and open problems.