Research / Topics
Machine unlearning, knowledge editing and removing hazardous capabilities.
Audits subspace-projection defenses against LoRA backdoors, showing they suppress static attack success without erasing the trigger, so backdoors reactivate within a few epochs of fine-tuning.
Proposes spectral unlearning, a matrix-structured optimizer update for LLM unlearning that improves memorization removal, privacy and utility trade-offs over existing baselines.
Proposes Hallucination-Guided Unlearning, using model hallucinations to localize and edit MLP neurons for targeted forgetting, reporting better forget-utility trade-offs on TOFU and WMDP.
Proposes a prompt-free audit of concept erasure in diffusion models, measuring residual concept probability via path-space KL cost. The barrier predicts leakage of NSFW and object concepts and guides a repair.
A survey of mechanistic interpretability for AI safety that covers its foundations and methods, proposes an evaluation framework, and assesses how understanding internals supports editing, steering and monitoring.
Shows that unlearned models can match retrained models on standard metrics yet encode different features, and proposes a feature-recovery measure that predicts relearning deviations.
Proposes VKCR, a parameter-efficient method that unlearns visual-to-knowledge associations in multimodal reasoning models, suppressing sensitive information under designated images while preserving permitted access, with new benchmarks.
Adds auditability as a fourth axis for LLM unlearning, proposing GapWiden and GapVeil to make deletion verifiable; it finds no existing method meets all four axes.
Builds model organisms with narrow and broad misalignment triggers, then proposes PEARL, a GRPO variant with prefix-expanded rollouts that durably suppresses misbehaviour while preserving capability.
Shows that fine-tuning on harmless data the model believes is false induces broad misalignment, scaling with disbelief. Single-fact updates show the same effect, and content filters and persona vectors miss it.
Proposes an attack-free proxy for unlearning robustness, measuring hidden-state fragmentation between queries and anonymized paraphrases to reveal suppressed knowledge that behavioral metrics miss.
Proposes PINT, an HSIC-based LLM unlearning method that reduces token-representation dependence to remove memorized data, including poisons and hazardous knowledge, while preserving utility up to 70B.
Proposes ALKN, a task-vector method with dynamic gradient masking and adaptive intensity for continual LLM unlearning, reporting better forgetting and utility retention than baselines.
Shows that converting high visual-pressure forget examples to text-only form improves unsafe-behaviour unlearning in vision-language models, with a per-sample routing method (ModRoute) beating multimodal or text-only unlearning.
Proposes SHIFT, an LLM unlearning method that extrapolates a shrink-and-heal lookahead toward the retrained oracle; it beats eleven baselines on TOFU and MUSE and resists relearning attacks.
Proposes KARU, an RL unlearning method that targets query-to-knowledge associations, with token-level rewards and retain sampling, to forget LLM knowledge while preserving general utility.
Proposes Continual Orthogonal Projection, a gradient-projection method that keeps repeated LLM unlearning requests from interfering, preserving utility and forgetting on TOFU and RWKU.
Proposes Safin-1.5, an LLM with a safety architecture, a discovery-evaluation-repair loop for recursive safety improvement, and removable hazardous-knowledge experts; claims it outperforms frontier models.
Introduces LoCo, a model-agnostic method to localize where and when visual concepts emerge in next-scale autoregressive image and video models, and applies it to concept erasure, personalization, and adversarial injection.
Derives closed-form output-space gaps for gradient-based unlearning in linear regression, extended to nonlinear models, and proposes leverage diagnostics for auditing forget sets.
Shows few-step distilled text-to-image models are less safe and resist concept erasure more than their teachers, and proposes DICE, a distillation-aware erasure method that cuts unsafe generations.
Position paper arguing machine unlearning should target statistically guaranteed output-space control of harms like toxicity and leakage, not parameter-space similarity to retrained models.
Tests whether filtering viral proteins from ESM-2 pretraining data selectively degrades hazardous capability; finds it moderately effective but recoverable by cheap fine-tuning.
Proposes ConceptACP, a diffusion-model unlearning method that uses sparse autoencoders and cross-attention block localization to remove harmful concepts efficiently while preserving image quality.
Shows that approximate unlearning in LLMs leaves recoverable traces in pre- and post-unlearning snapshots, and proposes TRACE, which reconstructs unlearned training text without prior data knowledge.
Proposes TaxoSAE, a stack of sparse autoencoders that encodes is-a taxonomies into LLM feature space, improving hypernym readout of unseen concepts and enabling more precise knowledge suppression and editing.
Introduces AfterDeleteBench, a behavioral audit showing LLM agents can recover deleted secrets via derived memory such as summaries, and that provenance-aware closure eliminates the recovery in tested settings.
Proposes Compositional Concept Unlearning, a query-side method that suppresses unwanted concepts in retrieval-augmented generation by editing the query, reducing target hits without raising control-query exposure.
Proposes Orthogonal Boundary Unlearning, a subspace-constrained method to remove hazardous dual-use capabilities (cyber, pathogen) from LLMs while preserving benign scientific knowledge, resisting priming attacks and relearning.
Proposes GRAFT, which confines weight updates from monitor-flagged reward hacks to a removable adapter, so ablating it prevents hacking, including hacks the monitor cannot detect, in small-scale experiments.
Proposes REX, which relocates hazardous knowledge into removable MoE experts so excising them removes WMDP-Bio knowledge while preserving general capability and resisting fine-tuning recovery.
Shows that unlearning in multi-agent LLM teams can leak forgotten information through combined messages, and proposes Aggregation-Conditioned Unlearning, which reduces recovery on TOFU and is tested on WMDP-bio.
Proposes EMDU, an anchor-first KL unlearning method for masked diffusion language models that fixes two failure modes of MDU, forgetting more on TOFU and RWKU while preserving retain utility.
Proposes CoLU, a confidence-weighted method that unlearns by updating a single selected transformer layer, with a layer-screening step that cuts runtime and memory while improving forgetting and retention on TOFU and MUSE.
Argues that behavioural forgetting is insufficient for privacy-oriented LLM unlearning and shows through cross-lingual querying that forgetting varies across access paths, so evaluation should test recoverability across them.
Proposes FIRM, a single-round Fisher-informed server-side merging method for federated LLM unlearning that avoids magnitude explosion and preserves retained utility on WMDP, TOFU and MUSE benchmarks.
Proposes Update-Matched Unlearning, which uses a relearning-matched reference so LLM unlearning resists harmful relearning while preserving the ability to learn new tasks.
Identifies three evaluation pitfalls in multimodal LLM unlearning benchmarks, introduces GEMMU with real-world knowledge and a fairer protocol, and finds substantial generalization gaps in existing methods.
Proposes RA$^2$, a latent-space LLM unlearning method that redirects forget-token states toward retain-supported representations, preserving benign reasoning near the forgotten domain on WMDP and MUSE.
Introduces Bemula, a benchmark testing whether LLM agent memory systems can truly forget user-requested facts without deleting unrelated memories or leaking them under extraction attacks; no evaluated system passes more than 1.5% of requests.