Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Shows that synthetic markers in SDF documents make implanted false facts detectable by linear probes, and that reducing them makes subtle false beliefs harder to distinguish from pretraining knowledge.
Patches attention heads and MLP chunks from a base model into a medical-misalignment fine-tune to localize and repair unsafe advice, finding a direction that removes but cannot install the behavior.
Rescaling a single German-specific parameter subcomponent from a parameter decomposition removes German ability in a 67M model with less English damage than LoRA at low data budgets, though it still hurts Italian.
Proposes Sparse Concept Anchoring, a training-time regularizer that reserves dimensions for chosen concepts so they can be suppressed or permanently ablated, tested only on a toy colour autoencoder.
Probing study finding that synthetic document finetuning on false facts shifts related beliefs, lowers truth separability in activations, and pulls confident priors toward the middle, in small Llama-3 models.
Shows that MSE reconstruction loss causes narrow SAE latents to absorb correlated features (feature hedging), proposes snap loss as a partial fix in toy models, and finds hedging likely in LLM SAEs.
Explains MONET, a transformer MoE architecture with about 250k monosemantic experts that aims for interpretability by design and allows removing domains or toxic behaviour by disabling experts.