AI research and discussion on LessWrong. www.lesswrong.com
Proposes GRAFT, which confines weight updates from monitor-flagged reward hacks to a removable adapter, so ablating it prevents hacking, including hacks the monitor cannot detect, in small-scale experiments.
Proposes grafting: run synthetic document fine-tuning on a base checkpoint and add the weight difference to the post-trained model, which preserves coherence and preference decisiveness better than native SDF for model organisms.
Empirically evaluates inoculation adapters, prompts and soft prompts on toy trait datasets, finding learned methods overfit, are less backdoor-susceptible, and fail at factual unlearning.
Analyzes inoculation prompting and inoculation adapters as alignment interventions against reward hacking and emergent misalignment, arguing they suppress rather than unlearn traits and proposing mechanistic hypotheses and experiments.
Argues that behavioural unlearning is not robust to relearning because capabilities stay in the weights, and proposes distillation into a fresh student as a safety intervention that drops latent unsafe structure.
A monthly roundup of AI safety papers covering pretraining data filtering against harmful finetuning, reward hacking generalizing to misalignment, covert sandbagging against CoT monitors, safe-completions training, and probe-based defenses against cipher attacks.}