Shows that reply length decides whether user pushback helps or harms LLM answers, probes hidden states to predict caving, and argues sycophancy evals should allow full replies and report self-correction.
Traces sycophancy across LLM training stages and finds SFT establishes most of it, not preference alignment, using interpretability probes on training data to support the analysis.
Introduces mechanistic shaping, which trains models through their internal representations to be right for the right reasons, reducing sycophancy, reward hacking and confounder use, and studies effects on monitorability.
Proposes VIAS, a method that builds value-isolating activation steering vectors so LLM agents comply with multiple organisational values, improving compliance over prompting alone on the PROTECT dataset.
Localizes value-sensitive attention heads in LLMs via probing, shows ablations change value behaviour, and proposes inference-time activation steering for single or combined value alignment without fine-tuning.
Finds a linear sycophancy-capitulation direction emerging at 7B+ in two model families; ablating it reduces answer reversals, with probe quality predicting causal effect.
Trains 134 personality LoRA adapters on Qwen3.5-4B and finds weight-space structure resembling the Big Five, with a CharIn score predicting persona shifts from training data before training.
Analyzes why safety-aligned LLMs over-refuse harmless prompts via representation and gradient geometry, then proposes GOAL, combining gradient orthogonalization and contrastive learning to reduce over-refusal while keeping safety.
Factorially tests replay and logit distillation for preserving LLM safety during sequential LoRA fine-tuning, finding distillation curbs erosion with less over-refusal and that base refusal-direction preservation is not a reliable safety proxy.
Proposes SimpleLoRIF, a gradient-attribution filter that removes benign-looking fine-tuning examples aligned with unsafe behavior, yielding SafeTulu, which cuts GCG attack success by up to 22%.
Proposes NEXUS, a hierarchical RL framework pairing neural skills with symbolic meta-policies, showing more interpretable, goal-aligned agents that resist reward hacking in games.
Tests whether the Assistant Axis predicts reward hacking in GPT-OSS-120B coding tasks; finds its signal is no better than many random directions, so assistant-ness is not clearly the driver.
Shows that a single-layer rank-1 LoReFT adapter can replace GRAFT's MLP forget adapter, matching reward-hack suppression while improving clean solve rate without an anchor environment.
Proposes REACT, a one-time activation steering intervention on early chain-of-thought states learned from safe and unsafe trajectories, which raises safe outputs on StrongREJECT from 29% to 82% on an 8B reasoning model.}
Shows that preference optimization builds a hidden shared hidden-state offset that is invisible to ranking metrics and can collapse generation, and proposes a representation-level signal to warn of this collapse.
Uses representation engineering with dynamic steering vectors to monitor and reduce reward hacking in realistic settings like tool calls and long contexts, with little capability cost and cross-domain generalization.
An introduction to LM psychology, arguing that psychology-level descriptions of language models help answer alignment questions like whether values are internalised, with tips on method and a reading list.
Argues that factual sycophancy is a failure of general in-context updating, using behavioral tests and causal mechanistic evidence, and warns that reducing sycophancy may also reduce useful updating.
Uses sparse autoencoders to find addressivity directions in post-trained LLMs and shows steering them amplifies or suppresses sycophancy, with the link strengthened by post-training.
Fine-tuning Qwen models on false facts they internally believe to be true induces emergent misalignment, scaling with the gap between stated and held beliefs, and screening methods fail to predict which data misaligns.
Identifies a cross-lingual set of safety-related neurons (safety kernel) in LLMs and uses them to constrain LoRA fine-tuning, substantially reducing multilingual jailbreak success rates while preserving general capabilities.
Builds 1,036 alignment-relevant concept vectors from 2M synthetic story pairs via activation differences, showing they detect concepts, steer behaviour, and reveal jailbreak representations across six models.
Traces attention heads behind cue-following sycophancy across post-training stages, finding a shared circuit from pretraining that RL strengthens and that can be edited to shift the cue effect.
Argues emergent misalignment from narrow bad-advice fine-tuning is stored mainly in system-prompt token keys and values, so perturbing the prefix restores aligned behaviour, suggesting a shallow, backdoor-like effect.
Shows that emergent misalignment fine-tunes of Qwen2.5-3B share a few attention heads whose ablation substantially reduces misaligned generations without hurting coherence.}
Evaluates four prompt- and activation-based inference-time steering methods for values in multi-turn conversations with opposing user and assistant values. The assistant's baseline bias often wins, and early effects often fade or reverse.
Traces where the intended goal is lost inside misgeneralizing visual navigation and manipulation agents, finding the depth predicts whether steering or rewiring can recover goal-directed behaviour without retraining.
Probes hidden states via vocabulary projection to show that models with prior harmful training retain more accessible hazardous concepts behind safety-aligned refusals, motivating internal audits.
Fine-tuning on polite gender and race stereotypes makes models less safe and less truthful on unrelated tasks; a stereotype direction in activations can control and monitor its acquisition.
Shows that running recurrent-depth LMs with fewer loops removes refusal behaviour while harmfulness stays linearly readable, and that randomized-depth safety training mitigates this.
Introduces alignment generalization prediction, showing activation-based value representations predict how fine-tuning on one value changes behavior on held-out values far better than text descriptions.
Introduces GUISE, a cross-language benchmark showing narrative wrappers bypass LLM refusals, and AXIS, a preference-optimisation defense using refusal-direction alignment that improves safety and usability.
Identifies refusal circuits in pretrained LLMs, traces how they change across alignment checkpoints, and proposes a circuit-confined alignment method that improves safety while reducing over-refusal and preserving utility.
Proposes a disentangled semantic-value interface for steering an LLM's normative priorities with fewer side effects, improving semantic preservation and reducing benign refusals at comparable value alignment.
Proposes CIA, a metric for agreement between LLM chain-of-thought and internal computation detected by interpretability tools, finds limited alignment, and shows post-training can improve faithfulness.
Proposes fine-tuning LLMs against activation probes for harmfulness and dishonesty, finding continuously updated probes improve safety, resist jailbreaks, and keep concepts linearly readable for oversight.
Shows that aligned fine-tuning data can induce misalignment in other contexts (context confusion), with a mechanistic explanation via shared representational shifts, and argues for post-training alignment evaluation.
Uses training data attribution to score how much each harmful fine-tuning example drives emergent misalignment, and shows score-based filtering can strongly amplify or reduce EM across model families.
Shows LLMs readily accept wrong answers attributed to verified sources, more than to users, and that internal source-deference directions can be ablated or steered separately from user agreement.
Shows that harmful fine-tuning removes LLM refusal, that layer-freezing and singular-direction repairs can be bypassed by adaptive attackers, and proposes five checks for defenses.