Neuron Atlas - Goodfire
Presents an interactive Goodfire tool for exploring language model neurons through overviews, mode landscapes and circuits; the abstract gives no explicit safety motivation.
Research / Sources
Goodfire
Understanding and steering neural networks. goodfire.com
Focus All
Topic All
Date Any time
Neurons as an Interpretable Basis
Argues that SwiGLU MLP neurons in language models are sparse, concisely describable, and predictable units of computation, and builds subcircuit and mode analyses plus automated neuron descriptions.
Shaping and Monitoring Models Through Their Internal Representations
Introduces mechanistic shaping, which trains models through their internal representations to be right for the right reasons, reducing sycophancy, reward hacking and confounder use, and studies effects on monitorability.
Neural Networks Are Already Clean
Argues that polysemanticity and superposition were misread as arbitrary packing, that networks' own neurons and circuits are legible, and that mechanistic interpretability is more tractable than feared.
Accelerating Interpretability: A Human Genome Project for AI
Goodfire essay proposing a cross-industry "Human Genome Project for AI" to reverse-engineer frontier LLMs, using agentic experiment-running to scale interpretability research for understanding and aligning AI.
Goodfire and Baseten partner to bring frontier safety to open models
Goodfire and Baseten announce Project Beacon, deploying white-box activation monitors to detect unsafe agentic behaviour in open models in real time, cutting judge costs about 50x.
Training and Deploying Production Cyber Monitors on Kimi K3
Builds probe-based cybersecurity monitors for Kimi K3 and GLM 5.3 that flag suspicious agent exchanges for an LLM judge, matching its recall at about 50x lower cost.
Better biosecurity monitors for AI agents via protein embeddings
Proposes sequence-aware biosecurity monitors using protein embeddings to flag dual-use agent tasks, reporting fewer refusals than frontier safeguards and better robustness to paraphrasing and fragmentation attacks.
We can and must solve alignment
Goodfire argues that interpretability is the bottleneck for technical alignment and outlines its plan to understand, monitor, and shape model behaviour.
A practical guide to sparse autoencoders (SAEs)
A practical guide explaining what sparse autoencoders are useful for, when to use probes or other alternatives, and how to train a good SAE on a model.
Models know when they’re reward hacking — and we can catch them at scale
Goodfire finds models have an internal signal accompanying reward hacking, and trains activation probes that detect it in real time, outperforming chain-of-thought monitors.
How to build fast, efficient monitors for AI models using probes
An introductory guide to activation probes as low-latency monitors that read model internals to detect unsafe agent behaviour, covering when and how to build them.
AI Safety Still Needs Great Engineers
Career essay arguing that AI safety needs strong software engineers to deploy safeguards like inference-time guardrails at scale, alongside alignment research. No technical results.
Announcing Goodfire Research Grants
Goodfire announces $1M in free access to its interpretability agent Silico for academic and nonprofit labs working on interpretability, alignment and safety. It is a funding announcement, not a research result.
Forking Fast: Efficiently Estimating Uncertainty Dynamics in Text Generation
Makes resampling analysis of LLM reasoning uncertainty cheaper by smoothing low-sample rollouts, finding uncertainty dynamics converge to stable patterns so far fewer samples are needed.
Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizers
Introduces Block-Sparse Featurizers that decompose vision model activations into multidimensional subspaces, finding more parsimonious interpretable features that enable fine-grained steering.
Meandering on Manifolds: The Neural Geometry of Stories Over Time
Studies how LLM activations trace trajectories on emotion manifolds as they read stories sentence by sentence, combining static geometry with temporal dynamics. Interpretability work with no explicit safety motivation.
Predictive Data Debugging: Reveal and Shape What Your Model Learns, Before You Train
Predicts which behaviours DPO will amplify or suppress from a preference dataset before training, traces them to responsible data, and proposes reshaping the data to prevent undesired effects.
Logits as a new monitor for evaluation awareness
Builds a logit-based monitor that estimates the probability of eval-aware sentences in a chain of thought, outperforming LLM-judge monitoring with 10-100x fewer rollouts on Kimi K2.5 and Qwen 3 32B.
Can SAEs Capture Neural Geometry?
Shows that sparse autoencoder features capture curved neural manifolds only partially, identifies three ways lines represent curves, and proposes an unsupervised pipeline for discovering concept manifolds.
A Geometric Calculator Inside a Neural Network
Goodfire finds Llama 3.1 8B uses a general-purpose addition module operating on circular Fourier representations of numbers, reused across addition-like tasks. The work is not safety-motivated.
Predicting Rare LLM Failures with 30× Fewer Rollouts
Proposes Logit Path Extrapolation, which estimates how often Qwen 3 4B complies with rare harmful requests using 30× fewer rollouts by extrapolating from a less safe variant.
The World Inside Neural Networks
Goodfire essay arguing that curved geometric structure in neural network representations is key to understanding and controlling models, citing examples across language, image and genomic models.
Steering Along Manifolds to Control Neural Networks
Proposes steering neural networks along curved manifolds in representation space rather than straight lines, using days-of-the-week concept geometry, and finds this gives better behavioural control.
Paper Summary: Interpreting Language Model Parameters
Introduces Adversarial Parameter Decomposition to split a 67M-parameter language model's weights into interpretable pieces, recovering algorithms and subnetworks, and enabling edits without training.
Interpreting Language Model Parameters
Introduces adversarial parameter decomposition to split language model weights into interpretable subcomponents, analysing attention circuits and editing behaviour; safety motivation is implicit in the interpretability framing.
Verbalized Eval Awareness Inflates Measured Safety
Shows verbalized evaluation awareness is common across models, correlates with safer behaviour, and causally inflates refusals in Kimi K2.5. Removing prompt cues reduces it, undermining safety eval reliability.
Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Training
Uses probes to trace a harmful compliance behavior that emerges during DPO post-training of OLMo 2 to specific preference datapoints, then filtering or relabeling them cuts it substantially.
Explaining 4.2 million genetic variants with state-of-the-art, interpretable predictions
Goodfire and Mayo Clinic probe the Evo 2 genomic foundation model to make interpretable variant-effect predictions for all 4.2 million ClinVar variants. This is non-safety interpretability applied to genomics.
Reasoning Theater: Probing for Performative Chain-of-Thought
Uses activation probes to detect performative chain-of-thought, where models already know their answer but keep reasoning, finding it tracks difficulty and enabling early exit that saves up to 68% of tokens.
Interpretability Infrastructure at Frontier Scale: Harvesting Activations from a Trillion-Parameter Model
Goodfire describes engineering an inference-server fork to harvest about 3 billion activations from Kimi K2 Thinking, a trillion-parameter model, in one overnight run.
Features as Rewards: Using Interpretability to Reduce Hallucinations
Goodfire trains Gemma-3-12B with RL using lightweight probes on internal activations as rewards (RLFR), cutting hallucinations by 58% without benchmark degradation, while the probe remains usable as a test-time monitor.
Intentionally designing the future of AI
Goodfire essay arguing that interpretability should guide AI training so new models share human values, framing intentional design as a scientific alternative to opaque training.
Understanding, Learning From, and Designing AI: Our Series B
Goodfire announces a $150M Series B for its interpretability company, framing mechanistic understanding of models as necessary for safe, steerable AI. Mostly funding news.
Using Interpretability to Identify a Novel Class of Alzheimer's Biomarkers
Goodfire interprets Prima Mente's epigenetic model for Alzheimer's from cell-free DNA, finds fragment-length patterns dominate, and derives a simple biomarker classifier that generalizes to an independent cohort.
Stanford Guest Lectures: AP293 (Fall 2025)
Goodfire shares recordings of three Stanford guest lectures on mechanistic interpretability: causal methods, transformer computational motifs, and in-context learning. Safety motivation is not stated.
Understanding Memorization via Loss Curvature
Uses loss-curvature analysis to separate weight directions supporting verbatim memorization from general computation, then edits models to suppress memorized content while preserving reasoning.
Priors in Time: Missing Inductive Biases for Language Model Interpretability
Argues SAEs impose stationary, independence priors that clash with language model temporal dynamics, and proposes Temporal Feature Analysis, which separates predictable from novel components and outperforms SAEs on garden-path and event-boundary tasks.
Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering
Proposes a Bayesian account unifying in-context learning and activation steering as belief updates, predicting LLM behaviour shifts, but without a safety motivation.
Deploying Interpretability to Production with Rakuten: SAE Probes for PII Detection
Applies sparse autoencoder probes to detect PII in multilingual user inputs for Rakuten's AI agents, finding they generalize from synthetic to real data and are far cheaper than LLM-as-judge.