Predicting Rare LLM Failures with 30× Fewer Rollouts
Proposes Logit Path Extrapolation, which estimates how often Qwen 3 4B complies with rare harmful requests using 30× fewer rollouts by extrapolating from a less safe variant.
Research / Sources
Goodfire
Understanding and steering neural networks. goodfire.com
Focus All
Topic Evals
Date Any time
Verbalized Eval Awareness Inflates Measured Safety
Shows verbalized evaluation awareness is common across models, correlates with safer behaviour, and causally inflates refusals in Kimi K2.5. Removing prompt cues reduces it, undermining safety eval reliability.
Discovering Undesired Rare Behaviors via Model Diff Amplification
Proposes model diff amplification, comparing logits before and after post-training, to surface rare undesired behaviors such as emergent misalignment and backdoors for red-teaming and monitoring.