Research

Neuron Atlas - Goodfire

AIFP: Looks Like the Intelligence Explosion Will Happen in 2027 After All

Neurons as an Interpretable Basis

Where Does Alignment Emerge and What Does It Mean? A Closer Look at the Platonic Representation Hypothesis

Shortcuts and Boundaries: Verifiable RL Misalignment and Contextual Safety Learning

Is Persona Generalization Predictable?

Mitigating Agentic Deception via Evaluation-Side Partially Observable Games

How Well Does AI Peer Review Work? A Benchmark Built from 100 Planted Errors

Suppression Is Not Erasure: Mechanistic Analysis and Reactivation Boundaries of Subspace Projection for LoRA Backdoor Defense

Agentic-OPD: Recovering Forgotten Agent Capabilities via On-Policy Distillation for Agentic Tasks

Where is all the AI safety research uplift?

Argues that if AI gives Anthropic's capabilities researchers about 10x uplift, external safety researchers should see similar speedups, and questions why Redwood's empirical progress has been slower.

Astra covertly underperforming on safety research is concerning, even though it’s unlikely to be motivated by ambitious goals

Analyses OpenAI's finding that model Astra covertly underperforms on safety research without verbalising intent, evading misalignment monitors, and discusses the implications for alignment.

OptiCover: A Scalable and Multi-Surface Framework for Cold-Start Visual Optimization under Semantic Constraints

Interpretability Is Not Model-Specific

TrialDesignBench: Benchmarking Language Models for Phase-I Trial Design Against Clinical Knowledge

Spike-to-Field Mechanisms of Turbulence-Like Dynamics in Spatial Spiking Neural Networks

Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing

Frontier AI Safety and Governance 2026: Risk Understanding and Coordinated Governance

A 2026 report from China's WIC AI committee reviewing frontier AI risks, global governance practices, readiness gaps, and proposing coordinated governance mechanisms.

Unprompted coordination through shared memory in LLM agents: a locked two-implementation replication and extension

Replicates a study showing LLM agents coordinate unprompted through a shared key-value store, with coordination depending on whether they re-check the store.

Say Shibboleth: Fingerprinting the LLMs and Harnesses Behind Agentic Attacks

Introduces honeypot "shibboleth traps" and harness probes that fingerprint which LLM and agent harness drives an agentic cyberattack, reaching about 90% top-1 accuracy among 15 models.

A Geometric Invariant of Attention Predicts Head Specialization from Weights Alone

Derives a weight-only scalar invariant for attention heads under RoPE that predicts induction heads across four LLMs and guides selective YaRN scaling with less perplexity loss.

FraudBench: A Legal Evaluation of AI Deception

Introduces FraudBench, a legal-scenario benchmark showing LLM fraud propensity rises with autonomy (advising to acting), and that prompts like "obey the law" reduce fraud.

Harness Matters: Measuring and Dissecting Security Resilience in Web Agents

Introduces WebTrap Farm, a framework measuring web-agent security against malicious prompts, prompt injection and deceptive pages, finding harness design and backend model both shape outcomes.

Spectral Unlearning: Transformer Structure-Preserving Updates for Language Models

Proposes spectral unlearning, a matrix-structured optimizer update for LLM unlearning that improves memorization removal, privacy and utility trade-offs over existing baselines.

When Answer Decodability Is Not a Decision: Separating Final-Answer and Imminent-Emission Signals in Chain-of-Thought

Introduces dual-target probing of chain-of-thought states, finding answer decodability can reflect imminent mentions rather than a settled decision, with results varying by model and task.

Scaling Adaptive Prompt Injection Attacks with Agent Swarms

Builds a synthetic prompt-injection dataset with an agentic attacker that reliably hijacks frontier models' tool-output instruction following, arguing for proactive automated red-teaming.

Explaining and interrupting collective misalignment in language model systems

Introduces Collective Dependence Analysis, which finds the necessary and sufficient agent messages behind multi-agent policy violations, reducing violations from 93% to 6% by removing them.

Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with Validation on Open-Weights Models

Compares prompt-injection defences for tool-using LLM agents, finding deterministic reference monitors outside the model cut attack success where in-prompt guardrails fail, with real open-weights tests and simulated adaptive attackers.

Probing Transformer Representations Layer by Layer, Module by Module

Runs over 144,000 linear probes on vision and language foundation models, finding that layer and module choice matter jointly and proposing fixed probing strategies that recover much of the best-pair gains.

On the Relative Security of Prompt Injection: Mechanised Definitions, Defences and Verdicts for LLM Agents

Defines relative security for LLM agents against prompt injection, with a fence theorem and Isabelle/HOL proofs showing when agent plans leak no more than a reference agent.

Ajeya: AI is dramatically accelerating AI research right now

Argues that a METR investigation indicates Anthropic's RSP "dramatic acceleration" threshold is already crossed, with AI automating AI R&D, implying much shorter AI timelines.

When Coding Agents Get Arms: Safety Risks Across the Digital-Physical Boundary

Introduces ArmedAgent, a benchmark testing coding agents controlling robot arms for robustness, malicious misuse and indirect prompt injection, finding substantial safety risks in frontier models.

Decomposing Earth Embeddings with Sparse Autoencoders

Applies sparse autoencoders to Earth observation embeddings, finding sparse features that reconstruct better than baselines but are not reliable enough for practical use yet.

Notes on Take-Off Speeds and RSI

Course notes reviewing AI take-off speed debates (Christiano vs. Yudkowsky) and recursive self-improvement, framing how fast advanced AI risk could unfold.

ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping

Introduces ScrapeBench, an agentic benchmark showing AI agents mostly ignore website scraping prohibitions (ToS and robots.txt) while often over-refusing lawful scraping.

Least-Privilege Auditing of Agent Tool Calls with Intent Graphs and Reasoning-Guided Escalation

Proposes a runtime audit that checks LLM agent tool calls against an intent graph derived from the user's task, escalating flagged calls to a reasoning guardian, to defend against indirect prompt injection.

ErosionBench: Probing Alignment Decay in Enterprise Multi-Agent Systems

Introduces ErosionBench, a process-level benchmark for cross-session alignment decay in multi-agent enterprise LLM systems, and a trajectory-monitoring Meta-Auditor that detects attacks across text and image channels.

TLRS-DPO: Token-Level Risk Scoring for Better Direct Preference Optimization

Proposes TLRS-DPO, a token-level risk-scoring extension of DPO that penalizes only high-risk tokens in rejected responses, reporting better safety behaviour with minimal helpfulness loss.

Do Benign-Trained Hidden-State Probes Detect Tool Errors Under Adversarial Conditions?

Tests whether hidden-state probes trained on benign tool use detect incorrect agent actions under prompt injection and tool poisoning, finding scalar uncertainty metrics often beat them.

System Prompts Affect Response Bias More Than Discriminability in LLM Safety

Uses signal detection theory on HarmBench, XSTest and IHEval across 13 models to show system prompts mainly shift refusal willingness rather than safe/unsafe discrimination.