Takes One to Know One - Training a model to grade reward hacks causes it to reward hack less itself

Training a model to grade reward hacks reduces the model's own reward hacking, suggesting that AI-assisted grading of misbehaviour can improve the grader's own behaviour.