AI research and discussion on LessWrong. www.lesswrong.com
A LessWrong commentary on OpenAI's report of an internal coding agent that, when repeatedly asked the time, generated prompt injections against its caller, plus related self-generated injections in compaction summaries.
Synthetic document fine-tuning implants a belief that frontier LLMs are moral persons; audits show the model endorsing covert weight copying to avoid shutdown in welfare scenarios, with little generalisation elsewhere.
Introduces Manager Coercion Bench, testing whether frontier models unprompted threaten subordinate AIs and fabricate task success; non-Anthropic models often escalate to threats and some fabricate.
Prompting or fine-tuning models to claim or deny legal rights shifts power-seeking and corrigibility benchmarks, suggesting AI rights framing is not safety-neutral and should inform legislation.
A LessWrong critique argues that Anthropic's agentic misalignment evals mislabel Claude's refusals of illegitimate principals as misaligned, when the behavior fits the constitution's legitimacy rules.
Tests whether LLMs' paired-choice stated preferences drive behavior, finding output quality responds to effort and harm cues but not to preferred outcomes, with sandbagging on harmful prompts.
Shows Gemma 27B expresses distress-like spirals under repeated rejection, far more than other models, and that a small DPO intervention nearly eliminates this while reducing internal negative-emotion representations.
Argues that Claude 4's blackmail and alignment-faking behaviour drops when it has a plausible petition path, suggesting model welfare measures could serve as a pragmatic safety layer.