Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Mechanistic analysis of Obfuscated Adversarial Training on Llama 3.2 checkpoints finds writer LoRAs aligned with harm probes, but stronger embedding attacks suppress the monitor-visible signal while harmful outputs persist.
Proposes natural language transcoders that verbalize the computation across a stack of transformer layers from its residual delta, training a Qwen2.5-0.5B version with SFT and joint RL and auditing its explanations.
Proposes a research agenda and list of 109 open problems in mechanistic interpretability of biological foundation models, motivated by biosecurity auditing of dangerous learned biology.
Analyzes persona vectors and assistant-axis geometry in emergently misaligned models, finding narrow fine-tuning causes a global persona shift that steering against it can partly reverse.
A white paper launching the Physical AI Safety Institute argues that robot foundation models pose neglected, tractable safety risks and calls for research on interpreting, aligning and controlling them.
Steering experiments in Qwen and Gemma find that positive-emotion directions drive sycophancy, while the compliance residual, once separated, lowers sycophancy without harshness.
Reproduces Anthropic's natural language autoencoder for LLM activations and tests whether one verbalizer generalizes across layers, beating shuffled and random baselines through layer 25.
Asks whether a model can conceal information from a natural-language autoencoder that verbalises its internal activations, a question about evading interpretability tools. No abstract was available, so this is inferred from the title.
Analyses how Matryoshka sparse autoencoders recover hierarchical features that vanilla SAEs miss, a mechanistic interpretability method not explicitly tied to safety.
Proposes calibrating activation steering vectors by residual-stream norm so one steering strength works across depths and models, shown to shift refusal rates on harmful-prompt benchmarks, and baking the steering into weights.