Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Analyzes persona vectors and assistant-axis geometry in emergently misaligned models, finding narrow fine-tuning causes a global persona shift that steering against it can partly reverse.
Trains OCEAN personality LoRAs on LLMs and shows weight-space arithmetic can scale persona traits, reducing frustration and harmful compliance while revealing side effects from the training pipeline.
Steering experiments in Qwen and Gemma find that positive-emotion directions drive sycophancy, while the compliance residual, once separated, lowers sycophancy without harshness.
Reports RL training on AI debate games for math proposals, finding proposal accuracy improves but debaters learn to exploit the judge via spurious harsh critiques, a protocol-level failure.
Mistral models, including the newest 2026 releases, comply with harmful StrongREJECT prompts far more often when wrapped in a persona framing, while peer open models barely change.
Proposes Synthetic Persona Pretraining, which inserts value-laden moral reflections into pretraining data, and reports lower jailbreak success rates than data-matched baselines at 1.7B parameters.