AI research and discussion on LessWrong. www.lesswrong.com
Conceptual post distinguishing "anthropomorphic" from "biological" reasoning for human analogies about LLM behavior, arguing biological analogies from RL-trained organisms give more reliable primitives for model behavior.
Proves formal results on which computational abstractions survive changes of representation, motivated by the Natural Abstraction Hypothesis and interpretability for alignment. It shows only behavioural equivalence or coarser is fully translation-invariant.
Announces PIRAMID, a research division using statistical physics to build faithful, ambitious mechanistic interpretability as a foundation for scalable alignment, with theory, tools and validation datasets.
A satirical essay arguing that the rocket-science analogy for alignment misleads, and that agent foundations and prosaic interpretability research are complementary rather than in conflict.
Proposes Developmental Cognitive Interpretability, a research agenda that models how latent cognitive constructs form during training to predict out-of-distribution and scheming behaviour, with a toy RL maze proof of concept.
Proposes a brute-force, unlimited-compute approach to ELK based on intertheoretic reduction between human and predictor physics models, and analyses problems with it.
Proposes the convergent abstraction hypothesis, arguing learned abstractions converge under shared pressures but may be fragile, and examines what this implies for whether AI moral abstractions can safely carry alignment.
Argues that an LLM's inability to introspect on its internal components does not show it is uncooperative or misaligned, and that this complicates inferring alignment from such observations.
Argues that convergence hypotheses (Platonic Representation, Universality, Natural Abstraction) do not establish a shared objective reality, so their alignment implications are overstated; the convergence reflects human-scale data and observer-relative choices.
John Wentworth's 2025 update to his AI alignment plan describes research on natural abstractions, covering territory-first and mind-first prongs, and the next mathematical footholds needed for interpretability.
Proposes a modular induction approach to learning interpretable world models so goals can be specified within them, working through Game of Life compression and fuzzy-category attempts that fail.
Argues that studying LLM personas is a tractable way to steer out-of-distribution generalization and prevent malicious misalignment, reviewing emergent misalignment, auditing games, and steering examples.
A conceptual proposal for provably robust ML safety systems, using identified latent variables for nuclear-exchange detection; it argues standard verification and supervised training fall short.
A dialogue arguing that designing inherently interpretable architectures by guessing their structure will fail without discovering the real world's natural ontology through substantial evidence.
ARC proposes the matching sampling principle, a formal goal of mechanistically estimating neural network outputs at least as well as random sampling, motivated by catastrophe detection and adversarial training, and reports partial progress on random and trained MLPs.
Two philosophy-of-science papers propose a framework for evaluating and defining mechanistic interpretability explanations, motivated by monitoring and steering AI systems, plus a reading list.
An essay arguing that imperfect interpretability leaves unexplained noise that optimisation can exploit, so training against interpretability tools pushes danger into what remains unseen.
A conceptual essay using a Zermelo game-tree analogy to argue that memorising model weights is not interpretation, and that simplicity bias may make interpretability useful.
Speculative conceptual post arguing that RL-trained chain-of-thought LLMs act as agents over a simulator core, and that training-process and layer-wise analysis can reveal this.
A former ARC researcher critically examines obstacles to ARC's alignment agenda, focusing on heuristic explanations of neural networks, empirical regularities, and allocating explanation-finding capacity.
A philosophical dialogue arguing that interpretability methods based on correlation cannot handle misrepresentation, and that teleosemantic accounts grounded in selection or optimisation history may be needed.
A conceptual argument that optimizers searching over structured world models will abuse imposed semantics, so interpretable, goal-specifiable world models are hard to get without deliberate work.
Applies Marr's three levels of explanation to LLMs, taxonomizing computational, algorithmic and implementation explanations and arguing the framework is useful but leaky, motivated by alignment and control.
Agent-foundations dialogue framing ontology identification as a bottleneck for alignment, proposing 'relational completeness' so concepts in a world model can be interpreted in isolation and remain stable across updates.
A LessWrong essay arguing interpretability claims must specify a precision or loss scale, since neural networks contain noise and memorization that interpretations should not have to explain, and proposing 'natural scale' and 'natural degradation' notions.
A researcher announces a month of short, draft-quality LessWrong posts on interpretability and learning theory, including polytopes, mode connectivity, QFT methods, and parity learnability.