Logits as a new monitor for evaluation awareness
Builds a logit-based monitor that estimates the probability of eval-aware sentences in a chain of thought, outperforming LLM-judge monitoring with 10-100x fewer rollouts on Kimi K2.5 and Qwen 3 32B.
Research / Sources
Goodfire
Understanding and steering neural networks. goodfire.com
Focus All
Topic Deception
Date Any time
Verbalized Eval Awareness Inflates Measured Safety
Shows verbalized evaluation awareness is common across models, correlates with safer behaviour, and causally inflates refusals in Kimi K2.5. Removing prompt cues reduces it, undermining safety eval reliability.