A survey of mechanistic interpretability for AI safety that covers its foundations and methods, proposes an evaluation framework, and assesses how understanding internals supports editing, steering and monitoring.
Proposes an attack-free proxy for unlearning robustness, measuring hidden-state fragmentation between queries and anonymized paraphrases to reveal suppressed knowledge that behavioral metrics miss.
Introduces LoCo, a model-agnostic method to localize where and when visual concepts emerge in next-scale autoregressive image and video models, and applies it to concept erasure, personalization, and adversarial injection.
Proposes ConceptACP, a diffusion-model unlearning method that uses sparse autoencoders and cross-attention block localization to remove harmful concepts efficiently while preserving image quality.
Proposes TaxoSAE, a stack of sparse autoencoders that encodes is-a taxonomies into LLM feature space, improving hypernym readout of unseen concepts and enabling more precise knowledge suppression and editing.
Proposes Nullify, a training-free inference-time activation steering method that unlearns private knowledge from LLMs while a null-space constraint preserves utility on TOFU and MUSE.
Shows that synthetic markers in SDF documents make implanted false facts detectable by linear probes, and that reducing them makes subtle false beliefs harder to distinguish from pretraining knowledge.
Proposes formally certified mechanistic edits that remove a harmful skill while provably preserving another over input regions, and shows that no finite black-box test can certify removal.
Audits the released VPD weight decomposition and finds its causal-importance labels break down under aggregation and deletion, causing output drift that undermines safe component editing.
RESCUE localizes sparse circuits behind LLM reasoning errors using RL-refined masks, then fine-tunes only those parameters, raising math and medical QA repair accuracy substantially while preserving other capabilities.
Trains a Reader model to describe frozen weight updates in natural language, then uses its gradients to guide edits that raise harmful-prompt refusal and some reasoning and tool-use scores.
Introduces a single sparse autoencoder shared across U-Net blocks and denoising steps in SDXL, and uses its features to suppress NSFW concept generation while tracing how concepts move through the network.
Introduces L2W, a method that localizes and surgically removes named semantic targets such as model identity claims and refusals from LLMs while preserving other capabilities.
Introduces SCALPEL, a contrastive sparse autoencoder that extracts more selective features for representation-level unlearning of targeted information, outperforming NMF and standard SAE interventions on TOFU across three LLM families.
Shows that linear concept erasure fails under dense superposition and that networks hide rather than delete entangled features, recoverable by a scalar patch, offering a mechanistic account of LLM unlearning failures.
Proposes Circuit-Diff, which uses low-rank factual edits to find which features in cross-layer transcoder attribution graphs relate to the edited knowledge, and validates them with causal patching on CounterFact edits.
Proposes ARIA, a test-time unlearning method that gates access to forgotten knowledge using SAE-latent detectors without changing weights, and reports better forget-utility trade-offs and robustness to post-unlearning attacks on TOFU, R-TOFU and WMDP.
Proposes Key Path Identification, which finds causally linked SAE features to steer LLMs toward contextual knowledge in RAG conflicts, beating mass steering with fewer edits.
Proposes EraseSAE, which uses sparse autoencoders to find and erase unwanted concept features in text-to-video diffusion models, reporting precise removal with less quality loss than prior methods.
Trains small language models with graded disentanglement between biology and other knowledge, then shows that less entangled models lose less retained capability under standard unlearning methods, supporting entanglement as a cause of collateral damage.
Proposes J-Access, an internal-representation audit that predicts how quickly unlearned LLMs recover removed knowledge; optimizing the audit makes models hide knowledge from it while recovery worsens.
Patches attention heads and MLP chunks from a base model into a medical-misalignment fine-tune to localize and repair unsafe advice, finding a direction that removes but cannot install the behavior.
Proposes Weight-Adjusted Gradients to find a tiny set of LLM parameters whose modification causes collapse, and shows uses in MoE allocation, unlearning, quantization and knowledge editing.
Gives a closed-form, Transformer-compatible MLP construction for fact storage that achieves optimal capacity scaling with far fewer parameters, and shows the MLPs can be swapped in for modular fact editing.
Audits whether neuron-attribution selectors causally identify important rows, showing that zeroing attributed rows can install refusal on harmful topics and that rank stability is a poor proxy for causal validity.
LLM agents are tasked with inventing concept-erasure algorithms that remove target concepts from model activations more completely than existing methods like LEACE, including against nonlinear probes.
Shows that sparse autoencoder features reliably detect concepts in diffusion models but cause artifacts when used for steering, and proposes detection-based patch replacement for cleaner object erasure.
Rescaling a single German-specific parameter subcomponent from a parameter decomposition removes German ability in a 67M model with less English damage than LoRA at low data budgets, though it still hurts Italian.
Shows that knowledge-edited facts in LLMs are often suppressed rather than erased and resurface under adversarial prompting, with mechanistic analysis explaining why edits are bypassable.
Uses gradient-based training-data attribution on OLMo3-7B to map which pretraining corpus regions support social versus STEM reasoning, with targeted unlearning as partial causal validation.
Extends the TOFU unlearning benchmark to five languages and finds unlearning transfers unevenly across languages, mostly suppresses rather than erases knowledge, and can be partly reversed with a steering direction.
Shows that projecting task vectors onto SAE feature subspaces discards most edit energy, and proposes using SAEs to select layers for raw task-vector edits that improve math accuracy.
Localizes toxicity to specific layers and neurons via activation differences, then suppresses it through retraining-free inference scaling or rank-one weight edits, with consistent toxicity reduction across five LMs.
Proposes the Unlearning Depth Score, an activation-patching metric that detects residual knowledge hidden in LLM internals after unlearning, and finds it outperforms 19 other metrics across 150 unlearned models.
Shows that 4-bit quantization undoes LLM unlearning of hazardous knowledge and proposes MANSU, a circuit-attribution null-space unlearning method whose forgetting survives quantization.
Proposes Sparse Concept Anchoring, a training-time regularizer that reserves dimensions for chosen concepts so they can be suppressed or permanently ablated, tested only on a toy colour autoencoder.
Proposes GUARD-IT, a training-free method that unlearns targeted content at inference time via gated activation redirection, evaluated on TOFU, MUSE and WMDP with quantization and jailbreak robustness checks.
Shows that residual memorization traces left after LLM unlearning can be detected by probes, and proposes probe-geometry alignment to erase them below chance without measurable capability loss.
CausalDetox uses causal mediation to find attention heads necessary and sufficient for toxic generation, then steers or fine-tunes them to reduce LLM toxicity while preserving fluency.
Uses causal tracing to show rule-level knowledge is distributed across transformer layers, and proposes Distributed Multi-Layer Editing (DMLE), which improves rule editing over fact-level baselines.