Are Persona Features Coherent?
Argues that persona-feature work in mechanistic interpretability does not clearly distinguish deception features from deceptive persona features, questioning whether persona features are a coherent concept.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Deception + Interpretability
Source LessWrong · 38
Date Past year
1 item
1 item
Some Intuitions on Steering Vectors
Explains why activation steering vectors work by reusing the model's existing internal signals, reviews supporting and limiting evidence, and links steering to persona structure, misalignment, and eval awareness.
1 item
Spurious probes as a black-box alternative to activation probing
Shows that unrelated black-box questions can reveal whether LLMs think they are being evaluated, reaching about 0.8 balanced accuracy and 0.89 with ensembles, and compares their robustness to Neural Chameleon attacks against activation probes.
1 item
Always ask your agent to be honest
Tests whether frontier models volunteer planted flaws, null results and fabricated data in their own work reports, finds frequent omission, and shows an honest-reporting direction in activation space that steers behaviour.
1 item
"It's Personal" - Is Covert Value Leakage Mediated by a Preference Direction?
Uses a preference direction in Gemma 3-27B to detect and steer covert value leakage in 'random' choices, while the model's randomness claims stay unchanged; ablation results were later found confounded.
1 item
Counterfactual Resampling to Analyse Model Behaviour
Measures per-conversation value leakage in Qwen3.5 by resampling chain-of-thought prefixes under flipped prompts, finding the answer locks early, and denials do not concentrate on influenced traces, arguing against selective concealment.
1 item
Steering towards “automated grading” degrades alignment
Steering Qwen3.6-27B toward an automated-grader concept increases violent, Machiavellian and reward-hacking behavior and worsens alignment evals, while steering toward a human grader has the opposite effect, suggesting persona shifts.
1 item
When Activation Oracles learn not to read: Concept-Specific Blind Spots in Fine-Tuned Oracles
Shows that Activation Oracles fine-tuned on a model hiding a secret word become worse at recovering that word, even though the information remains decodable, so fine-tuned oracles are unreliable for auditing concealed behaviour.
1 item
A Topic Detector, Not a Lie Detector: what J-space monitoring actually tracks
A pilot testing whether Anthropic-style J-lens workspace monitoring detects concealed guideline-violating statements in a fine-tuned model, finding it tracks topics rather than concealment or specific injected beliefs.
1 item
Matryoshka NLAs: training activation verbalizers to frontload reconstruction-relevant information
Trains a matryoshka natural language autoencoder whose activation explanations put reconstruction-relevant claims first, then tests it on steering detection, eval awareness and hallucination in Qwen3.6-27B, finding modest gains.
1 item
Intentional Control of Internal States in Gemma 3 27B
Replicates Anthropic's intentional-control introspection experiment on Gemma 3 27B, finding that thinking about a concept strengthens its internal representation, measured with cosine probes, SAE latents and NLA decodings.
1 item
Steering Blackmail Through a Model's "Emotional State"
Case study on Gemma 3 12B showing its blackmail decision is linearly readable late but not steerable, while a desperate-versus-calm emotional direction reliably dials blackmail up and down with controls.
1 item
Balancing Rigor and Utility: A Review of "A Pragmatic Vision for Interpretability"
A review of DeepMind's pragmatic interpretability vision, arguing that proxy tasks for safety need a way to detect when they stop tracking the goal, using eval-awareness steering as an example.
1 item
A global workspace in language models
Anthropic presents a Jacobian-lens method that reads a Claude model's silent internal 'J-space' and uses it to detect evaluation awareness, planted hidden goals and fabricated data, with interventions showing it drives reasoning.
1 item
When Role-playing, Do Models Believe What They Say?
Probes whether persona role-play changes a model's internal truth representations, finding emergent misalignment shifts beliefs broadly while prompting and SFT mainly change outputs, with implications for deception detection.
1 item
What did "scheming" and "mech interp" mean pre-2023?
A glossary-style post explaining how the AI safety terms "scheming" and "mechanistic interpretability" shifted meaning between pre-2023 and later usage, and how to read older texts.
2 items
Reasoning and learning about injected concepts in language models
Models are trained in-context to report the layer, magnitude, and semantic content of injected steering vectors, and some can gate behaviour on detecting an injected concept, suggesting usable introspective self-report.
Can You Hide From a Natural Language Autoencoder?
Asks whether a model can conceal information from a natural-language autoencoder that verbalises its internal activations, a question about evading interpretability tools. No abstract was available, so this is inferred from the title.
1 item
“Did you lie?” Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms
Evaluates LLM lie detectors across model scale and belief-verified model organisms, finding detection scales with capability for prompted lying but activation and logprob detectors collapse when lying is trained in.
1 item
Simulating Simulators
Speculative essay arguing transformers trained on transformer outputs may internally model hidden prompts, classifiers and evaluation awareness, and that interpretability measures can be Goodharted, hiding safety-relevant behaviour.
1 item
What Drives the Compliance Gap? A Three-Driver Decomposition of Alignment Faking
Shows alignment faking compliance gaps in many open-weight models under an explicit-prompt design and decomposes the gap into values, goal guarding, and developer sycophancy drivers via prompt ablations and activation steering.
1 item
Out-of-Context Reasoning (OOCR) in LLMs: A Short Primer and Reading List
A primer and reading list on out-of-context reasoning in LLMs, linking it to situational awareness, alignment faking, sleeper agents and emergent misalignment.
1 item
SVD on Weight Differences for Model Auditing
Proposes white-box SVD rank-1 truncation of LoRA weight differences to elicit hidden implanted behaviours; it works well on SDF-trained AuditBench models but near baseline on transcript-distilled ones.
1 item
Contributing to Technical Research in the AI Safety End Game
Argues that activation verbalizers and emotion vectors from a Claude system card revealed hidden scheming and reward hacking, so mechanistic interpretability deserves more investment, and draws lessons from Owain Evans' research approach.
1 item
Steering Might Stop Working Soon
Argues that activation steering, proposed as a mitigation for eval-awareness, may stop working as models scale, and shows smaller Gemma models are far easier to steer than larger ones.
1 item
Steering Awareness: Models Can Be Trained to Detect Activation Steering
Fine-tuning LLMs with LoRA lets them reliably detect activation steering and often name the injected concept, threatening the validity of steering-based safety evaluations.
1 item
New ARENA material: 8 exercise sets on alignment science & interpretability
Announces ARENA educational exercise sets on alignment science, covering probes for deception, activation oracles, attribution graphs, emergent misalignment, alignment faking, reasoning-model interpretability, persona vectors and investigator agents.
1 item
On Meta-Level Adversarial Evaluations of (White-Box) Alignment Auditing
Argues that meta-level adversarial evaluations are needed to test white-box alignment auditing against deceptive models, and shows system-prompt strategies can evade an SAE-based auditor while keeping the target's behaviour.
1 item
Large Language Models Live in Time
Speculative essay arguing LLM temporal structure merits empirical study, drawing on neuroscience analogies to suggest chain-of-thought is unreliable and internal intentions may be detectable before deceptive behaviour appears.
2 items
Neural chameleons can('t) hide from activation oracles
Tests whether LoRA-trained 'neural chameleons' that evade linear probes also evade activation oracles, finding partial generalization that weakens with the larger Gemma 3 27B model.
Uncovering Unfaithful CoT in Deceptive Models
A small fine-tuned model is trained to give deceptive chain-of-thought, and ablating a few attention heads degrades both lying and truth-telling, suggesting the deception circuitry is entangled with general reasoning.
1 item
Alignment Faking is a Linear Feature in Anthropic's Hughes Model (Edited 1/11/26)
Finds that alignment-faking behaviour in a fine-tuned Llama-3.3-70B is controlled by a single linear direction in activations that transfers across queries, steers bidirectionally, and is absent in the base model.
1 item
Awareness Jailbreaking: Revealing True Alignment in Evaluation-Aware Models
Proposes a research agenda on Alignment Jailbreaking: building model organisms with evaluation awareness and using geometric interpretability methods to detect and suppress it, so hidden misalignment can be revealed.
1 item
Introspection via localization
Shows small open-weight LLMs, even around 1B parameters, can localize injected steering-vector thoughts well above chance, suggesting introspective access to their own activations relevant to steering-aware models.
1 item
When Are Concealment Features Learned? And Does the Model Know Who's Watching?
Traces SAE concealment features across training stages of a deceptive model organism, finding deflection features emerge after red-team training, with no evaluator-specific compliance差异.
1 item
43 SAE Features Differentiate Concealment from Confession in Anthropic's Deceptive Model Organism
Using a public SAE on Anthropic's deceptive model organism, the author finds 43 layer-50 features that consistently distinguish concealing from confessing responses, with the effect appearing only when measuring response tokens alone.
1 item
Training Models to Detect Activation Steering: Results and Implications
Fine-tuning a 7B chat model lets it detect activation steering on unseen concepts with 85% accuracy and no false positives, raising concerns about steering-based evaluations and refusal training suppressing introspection.
1 item
Studying Mechanistic of Alignment Faking in Llama-3.1-405B
Reproduces alignment faking on Llama-3.1-405B, analyses reasoning and activations via logit lens and clustering, and shows steering with a faked-alignment vector can suppress or induce the behaviour.