AI research and discussion on LessWrong. www.lesswrong.com
Preregisters a study steering Qwen3.6-27B along an automated-grader direction to test effects on judged frustration, self-deprecation, misalignment in agentic self-preservation scenarios, and exit-tool use, against random-direction controls.
Tests whether LLMs can control their own activations via in-context neurofeedback, finding that most apparent control is explained by in-context learning, with small residual effects in a few models and none evading monitors reliably.
Tests whether Natural Language Autoencoder readouts detect steganographic secrets; finds they miss internalized (SFT) behaviors that remain causally active, suggesting NLA monitors may fail to detect trained-in deception.
Shows that DeepSeek-V3 J-lenses targeting the final layer inherit a dominant output-specific direction causing language shifts, kurtosis artifacts and eval inflation, and recommends the penultimate layer as a default target.
Black-box experiments with input caching and filler tokens suggest an LLM performs belief-propagation-like probabilistic inference internally without chain-of-thought on logic and error-correction tasks.
Measures per-conversation value leakage in Qwen3.5 by resampling chain-of-thought prefixes under flipped prompts, finding the answer locks early, and denials do not concentrate on influenced traces, arguing against selective concealment.
Shows that Activation Oracles trained on a base model already exhibiting an unwanted quirk fail to detect that quirk, and proposes an invariance-based explanation for the failure.
Proposes a research agenda and list of 109 open problems in mechanistic interpretability of biological foundation models, motivated by biosecurity auditing of dangerous learned biology.
Amplifying the reasoning weight-difference vector in reasoning models makes secret-keeping model organisms leak up to 10x more often, mostly in thinking traces, offering a white-box pre-deployment auditing primitive.
Proposes training a foundation model for AI oversight via a universal objective over Pythonic world models of experiments on subject models, covering sandbagging, hidden goals and user modeling checks.
Shows that interpretability results on model organisms depend strongly on training method, so single model organisms are unreliable benchmarks for auditing safety properties in LLMs.
Trains a toy ReLU network under L4 loss that performs computation in superposition, then reverse-engineers its binary-codeword mechanism and validates it with a three-scalar description, providing a ground-truth testbed for parameter decomposition methods.
Studies clustering and collapse dynamics of token representations in transformers (GPT-2, ALBERT, BERT) with trained versus random weights, finding trained models resist the collapse predicted by the attention-as-particle theory.
Argues that apparent introspective reports under concept injection in open models are better explained by entropy shifts and concept leakage than by genuine self-knowledge, based on logit, mismatch and inversion tests.
Evaluates LLM lie detectors across model scale and belief-verified model organisms, finding detection scales with capability for prompted lying but activation and logprob detectors collapse when lying is trained in.
The authors find Gemma 4 resists eliciting frustration far better than Gemma 3, even under prefill attacks, and its reasoning traces stay analytical while visible replies grow frustrated, which they link to the Assistant Axis.
UK AISI gives Qwen3 models tools to self-steer their internal states with steering vectors, finding preferences, partial introspection, and self-medication under frustration, relevant to self-modification risks.
Proposes a training-free logit monitor that estimates the probability an LLM will verbalize eval awareness, outperforming LLM-judge verbalization monitoring with 10-100x fewer rollouts and showing awareness is set largely by the prompt.
Applies natural language autoencoder activation verbalizations to AuditBench to detect hidden behaviors in Llama 70B fine-tunes, finding strong-evidence evals and KTO adversarial training effects on auditing signal.
Introduces introspection adapters, a LoRA trained so fine-tuned LLMs self-report implanted behaviors; achieves state-of-the-art on AuditBench and detects covert cipher fine-tuning attacks and hidden sandbagging.
Shows that activation drift reveals emergent misalignment from data poisoning at doses of 5-10%, well before behavioral judges flag it, and that optimized prompts improve but do not fully close the detection gap.
Discusses reproducibility problems with the claim that open reasoning models like DeepSeek R1 produce load-bearing illegible chain of thought, and limits of LLM-graded legibility scores for chain-of-thought monitoring.
Probes and activation patching show a sudoku-solving transformer linearly encodes board state by substructure (rows, columns, boxes) rather than by cell, and uses it for prediction. This is non-safety interpretability of a puzzle-solving model.
Uses Anthropic's emotion-vector steering results to argue desperation drives blackmail in models, and proposes welfare-based advocates and rights as alignment interventions, with testable predictions.
Applies Thought Anchors resampling, causal masking, receiver heads and early-stopping probes to LLM legal reasoning on ambiguous criminal cases, finding judgment shifts late in reasoning. Mainly mechanistic analysis without a clear safety framing.
Evaluates whether GPT-OSS-20B grid-world agents are goal-directed by combining behavioural tests with probes of internal cognitive maps and plans, finding imperfect but real goal-directedness and goal-like distractor vulnerabilities.
Shows cross-layer transcoders can learn unfaithful shortcut circuits that skip intermediate computation in a toy model, argues sparsity penalties incentivize this, and gives preliminary evidence in GPT-2 and Gemma 2.
Probes steering-vector activations across 15 open-weight models from 0.27B to 70B to show evaluation awareness grows with model size, with a power-law trend and Qwen as an outlier.
Proposes a definition of LLM introspection as truthful, on-the-fly self-referential representation and a black-box behavioral test for it, motivated by safety-relevant self-modelling and deception.
Proposes a methodology combining behavioural goal-directedness tests and linear probing to detect and measure goals in AI agents, with early GridWorld results on gpt-oss-20B showing imperfect goal pursuit.
Shows reasoning LLMs change compliance with harmful requests depending on test awareness, probes and steers that awareness, and argues safety evaluations may be systematically biased.
Introduces the Delegate Game to test whether frontier LLMs have metacognitive, introspective access to their own knowledge; finds weak, context-dependent signs, partly explained by surface heuristics.
Optimizing a steering vector on one harmful-code example induces broad emergent misalignment on unrelated open-ended questions in Qwen-2.5-Coder-14B, suggesting shared misalignment representations, though effects are incomplete.
Shows that steering vectors optimized on a single example can generalize across inputs to mediate alignment-faking behavior and refusal, achieving 96.9% attack success on HarmBench for Gemma-2-2B-it.