Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
Pretrains 30B LLMs with and without filtering subversion-relevant data, finding filtered models know far less about AI control subversion while retaining general ML knowledge, as a proof of concept for reducing misaligned AI risk.