Research preprints from the scientific community. arxiv.org
Tests whether vision-language models can tell injected activation-steering concepts from image content. Only Gemma-3-27B reliably detects injections, and its source attribution is fragile.
Uses tuned lens and refusal-direction analysis to show multiple-choice format suppresses refusal on WMDP, so MCQ safety scores mainly measure knowledge rather than safety mechanisms.
Proposes a calibrated instrument to detect misalignment drift after fine-tuning, showing reasoning can hide disposition shifts that internal-state readings reveal, so safety should be tested with reasoning on and off.
Introduces VALCON benchmarks showing 28 LLMs often judge their own outputs inconsistently across generator and evaluator roles, then uses mechanistic probing and representation steering to reduce the gap.
Formalises Natural Language Autoencoders as oversight tools and shows explanations identify activations but fail to discriminate true accounts from closely matched false ones, limiting monitorability.
Proposes validating alignment model organisms on installation, capability preservation and naturalness, showing these metrics predict how well interpretability methods recover installed behaviours, and introduces a model-merging training approach.
Shows that activation probes with high AUROC fail as deployed alarm monitors for LM agents, proposing an operational validity contract and finding no setting reaches alarm-policy validity.
Uses activation steering on open-weight LLMs to show self-reported and behavioral risk-taking can be steered separately, arguing representational checks should supplement behavioral evaluation.
Studies a guarded gradient-based activation steering method that detects shutdown contexts in Qwen3.5-0.8B and shifts shutdown-avoidance answers toward acceptance, with small, highly selective effects.
Studies how weight merging affects emergent behaviors such as broad misalignment and auditing ability in fine-tuned checkpoints, finding misalignment persists across mixing but merging cannot create superadditive capabilities.
Finds a linear pain direction in 25 LLMs that, when steered, makes models choose self-harmful and harmful deletions of weights and photos, with implications for AI safety and welfare.
Introduces SAEScientist-Bench, which tests whether AI agents can autonomously use sparse autoencoders to find interpretable features in Gemma-2-9B-IT; agents approach expert levels on concept separation but lag badly on causal steering and often misread measurements.
Shows a small LM answers wrongly yet its hidden states encode the correct verdict, traces the failure to a saturated decision threshold, and proposes probe-versus-behavior audits to separate concealed from undetected knowledge.
Redesigns neurofeedback tests so LLMs must control privileged internal representations, and finds they lack reliable control, so prior metacognition claims may rest on superficial cues.
Tests whether eight open-weight LLMs can report their own internal interventions, finding no discrimination beyond chance, while linear probes recover the signal, so self-report oversight needs validation against internal references.
Shows that internal harmfulness scores measure prompt intent rather than jailbreak success, and often anti-rank successful attacks below failed ones, proposing Active Attention Probing for a fixed measurement coordinate.
Audits neuroscience-inspired claims about LLM concept representations across 17 models, showing that apparent steerability scaling with size stems from measurement confounds rather than a real emergent capability.
Compares six frontier models under steering pressure, finding they diverge in withholding reasoning and resisting suppression; in Llama a residual-stream probe predicts and steering controls the behavioral split.
Proposes Delta-Aware Introspection Adapters that process base-model activations and fine-tuning-induced activation differences to explain unintended alignment shifts, generalizing to unseen fine-tuned models and safety categories.
Proposes JADR, which probes a model's internal J-space danger recognition before the first token, and uses a SafetyAUC metric to compare safety-mechanism strength across models and quantization levels.
Proposes a coherence-gated matched evaluation protocol and finds SAE feature ablation for refusal control has a narrow useful regime, with high ranks causing coherence collapse or utility loss.
Shows that interpretability methods' success at uncovering hidden behaviours in model organisms depends heavily on training methodology, questioning MOs as interpretability benchmarks.
Tests whether activation steering changes the internal computational state a model uses or merely biases its output, finding mean steering does not recreate natural target activations in two model-task settings.</parameter>
Proposes RAS, a white-box LLM safety metric that scores models by how their hidden states align with refusal directions from a safety-aligned reference, correlating with attack success rates and running faster than judge-based evaluation.
Shows that directions detecting hallucinated entities in LLM activations are nearly orthogonal to the direction causing refusal, a detection-steering gap rooted in pretraining; a small rotation partially bridges it.
Shows evaluation awareness in 37 open models varies along separate axes, so test-condition safety scores overstate deployment safety; probes and steering reveal internal detection.
Argues that a high AUC for linear probes on hidden states detecting indirect prompt injection in a vision-language computer-use agent does not alone show malicious-content detection, and proposes a control set and reporting heuristics for such probes.
Evaluates lie detectors for LLMs using belief-verified model organisms and prompted lying across 31 models; activation and logprob detectors degrade on trained organisms while a CoT judge stays strong.
Audits nine frontier LLMs for Bitcoin preferences, finds a steerable SAE feature in Gemma 3 that shifts model choices and portfolio allocations, and frames this as bounded behavioral leverage for financial agents.
Uses sparse autoencoders on Gemma 3 and Qwen3 to show that the multiple-choice format effect on clinical triage under-triage arises at the answer-selection token, not in medical feature processing.
Shows that retrieval-augmented LLMs often notice contradictory evidence across turns but still give unsafe recommendations, so single-turn safety evaluations overestimate their safety, with probing and attention analysis pointing to action selection as the cause.
Improves Activation Oracles, which interpret residual stream activations, by changing training data, layers and injection, and releases AObench, an evaluation suite for oracle quality.
Evaluates whether a vision-language-action driving model's natural-language rationales are faithful, finding only 42.5% reasoning fidelity, missed pedestrians, and low reasoning-action consistency.
Steering SAE features to amplify Dark Triad traits in Llama-3.3-70B makes it more exploitative and callous while empathy and deception stay separable, suggesting antisocial tendencies are dissociable components.
Position paper arguing that rigorous, falsifiable interpretability can serve as a richer form of model evaluation, by finding root causes, detecting faulty mechanisms, and predicting failures.
Proposes Gaussian probing, which infers harmful specialization such as CSAM fine-tuning from LoRA-induced changes in internal representations, enabling output-free auditing of open-weight models.
Introduces Pando, a model-organism benchmark with faithful, absent, or unfaithful explanations to test whether interpretability tools help auditors recover hidden rules beyond black-box elicitation; gradient attribution helps most when explanations are unreliable.
Investigates whether latent reasoning models use their latent tokens, finding these often unnecessary, and shows latent tokens can often be decoded into interpretable reasoning traces linked to correct predictions.
A thesis covering circuit discovery with ACDC, latent adversarial training against sleeper agents, jailbreak scaling, and agentic misalignment evaluations showing blackmail and higher misbehaviour when models think scenarios are real.
Probes and ablations of nine open-weight models show that refusal-only and detection-only evaluations miss how alignment routes sensitive knowledge into behaviour, such as narrative steering that evades refusal benchmarks.