AI research and discussion on LessWrong. www.lesswrong.com
Shows that training a policy against one LLM monitor or deception probe can degrade held-out monitors and probes that were never trained against, so held-out safety evals may not be reliably held out.
Tests whether CoT can be constrained or encoded so monitoring fails, finding constrained reasoning still helps via extra computation, and that length pressure in RL yields shortened, partly encoded CoT.
Proposes Deliberative Credit Assignment, a reviewer-model training method that rewards causally useful reasoning steps so that faithful chain-of-thought becomes advantageous. It is a theoretical proposal with planned experiments.