Tests whether filtering pretraining data can safeguard open-weight protein models against dual-use biological capabilities; finds it selective and effective but not durable against fine-tuning.
Proposes a certification framework giving high-confidence upper bounds on residual concept leakage after unlearning in text-to-image diffusion models, finding attack-based evaluations underestimate leakage by 16.2%.
Presents SafeTune, a library unifying methods for detecting and repairing safety drift in fine-tuned LLMs, including weight recovery, safety-constrained fine-tuning, unlearning and steering, with evaluation utilities and case studies.
Shows that abliterating refusal directions from open-weight models causes off-target shifts in optimism and confidence on a decision task, so 'uncensored' models behave as measurably different decision-makers.
Presents an automated contrastive pipeline that generates statistically validated hypotheses about behavioral side effects of interventions like distillation, knowledge editing and unlearning, recovering injected changes and surfacing unexpected shifts.
Proposes a dynamic framework that stress-tests LLM unlearning with multi-hop and aliased probes, showing forgotten knowledge often resurfaces, and uses activation analysis to explain why.
Shows that LLM unlearning of hazardous content can cause unintended losses of benign knowledge that standard benchmarks miss, and proposes a test-generation framework to expose these knowledge holes.
Taxonomizes twelve LLM unlearning methods and shows that multiple-choice WMDP metrics overstate hazardous-knowledge removal, proposing open-QA metrics that expose a utility-unlearning tradeoff and finer robustness failures.
Shows that common LLM belief metrics can be satisfied by simply prompting a model to pretend a false fact is true, and proposes harmfulness-based tests to separate real belief from role-play in safety experiments.
Filters CBRN weapons information from pretraining data, then pretrains models from scratch, cutting accuracy on a harmful-capabilities eval by 33% relative to baseline while preserving benign capabilities.
Pretraining from scratch on data filtered of dual-use biothreat text yields open-weight 6.9B models resisting up to 10,000 steps of adversarial fine-tuning, though filtered models can still use in-context information.
Shows that standard unlearning leaves sensitive information in reasoning model chain-of-thought traces and proposes R²MU, a representation-misdirection method that suppresses those traces while preserving reasoning ability.
Introduces OpenUnlearning, a standardized framework benchmarking 13 LLM unlearning methods and 16 evaluation metrics across TOFU, MUSE and WMDP, plus a meta-evaluation of metric robustness.
Critiques current LLM unlearning evaluations, finding they inject new information, vary across tasks and rely on spurious correlations, and proposes minimal-injection and downstream-awareness principles.
Introduces R-TOFU, a benchmark for unlearning private or copyrighted knowledge from reasoning traces in large reasoning models, showing answer-only methods leave residual knowledge in chain-of-thought and proposing a Reasoned IDK baseline.
Proposes UNCD, a cognitive-diagnosis framework and cyber benchmark that finely evaluates LLM unlearning of dangerous knowledge, and an agent that targets remnants to improve removal.
Proposes model tampering attacks (latent and weight modifications) as a more rigorous way to evaluate LLM dangerous capabilities, showing current unlearning can be undone within 16 fine-tuning steps.
Position paper arguing machine unlearning has inherent limits as an AI safety tool for dual-use knowledge like CBRN and cybersecurity, and mapping open problems in side effects, evaluation, and robustness.
Proposes probabilistic evaluation metrics for LLMs showing deterministic greedy evaluations overstate unlearning and alignment, and improves unlearning with an entropy-based loss and adaptive temperature scaling.
Shows that frontier LLMs can infer latent facts from scattered training data without in-context examples, so censoring dangerous knowledge from training data may not fully prevent it.
Proposes more robust metrics and a calibration method for evaluating LLM unlearning, addressing attack vulnerability and the unlearning-retention trade-off when comparing methods.
Releases WMDP, a public benchmark of hazardous bio, cyber and chemical knowledge in LLMs, and RMU, an unlearning method that reduces that knowledge while keeping general capabilities.