Understanding and steering neural networks. goodfire.com
Introduces mechanistic shaping, which trains models through their internal representations to be right for the right reasons, reducing sycophancy, reward hacking and confounder use, and studies effects on monitorability.
Uses probes to trace a harmful compliance behavior that emerges during DPO post-training of OLMo 2 to specific preference datapoints, then filtering or relabeling them cuts it substantially.
Goodfire trains Gemma-3-12B with RL using lightweight probes on internal activations as rewards (RLFR), cutting hallucinations by 58% without benchmark degradation, while the probe remains usable as a test-time monitor.
Goodfire essay arguing that interpretability should guide AI training so new models share human values, framing intentional design as a scientific alternative to opaque training.