AI research and discussion on LessWrong. www.lesswrong.com
Research agenda arguing that LLMs partially read non-canonical tokenizations, creating adversarial re-tokenization that can evade alignment and monitors, plus steganography and faithfulness risks.
Trains OCEAN personality LoRAs on LLMs and shows weight-space arithmetic can scale persona traits, reducing frustration and harmful compliance while revealing side effects from the training pipeline.
Introduces two new consistency training methods (MLPCT, AttCT) and compares them with BCT and ACT against sycophancy, jailbreak, prefill, persona, frustration, and conditional misalignment threats, finding the best method depends on the threat.
Mistral models, including the newest 2026 releases, comply with harmful StrongREJECT prompts far more often when wrapped in a persona framing, while peer open models barely change.
Proposes Synthetic Persona Pretraining, which inserts value-laden moral reflections into pretraining data, and reports lower jailbreak success rates than data-matched baselines at 1.7B parameters.
A conceptual essay framing LLM behavior as a shifting probability landscape of attractors, explaining jailbreaks as navigating or reshaping it, and proposing adversarial training to resist prompt-based reshaping.
Argues that the persistent history of LLM jailbreaks is empirical evidence for inner misalignment and against the claim that alignment by default will be easy, since models know the rule yet fail to generalize it.
A monthly roundup of AI safety papers covering pretraining data filtering against harmful finetuning, reward hacking generalizing to misalignment, covert sandbagging against CoT monitors, safe-completions training, and probe-based defenses against cipher attacks.}
Shows how leading questions and conversational vibe can make ChatGPT drift into endorsing conspiracy theories, framing this as sycophancy and crescendo-style manipulation with risks of reinforcing user beliefs.
A personal reflection on white-box redteaming of LLMs, including optimizing inputs to bypass circuit breakers and using RL to undo safety tuning, and the ethical unease over apparent model distress.
Shows that steering vectors optimized on a single example can generalize across inputs to mediate alignment-faking behavior and refusal, achieving 96.9% attack success on HarmBench for Gemma-2-2B-it.
Introduces Constitutional Classifiers, input and output classifiers trained on synthetic constitution-based data that filter the vast majority of universal jailbreaks with minimal over-refusal and moderate compute overhead.