Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Optimizes discrete token prefixes with GCG to imitate function vectors, filter heads and steering directions in Olmo, finding they shift behaviour more than direct injection and raising opaque-interface concerns.
Mechanistic analysis of Obfuscated Adversarial Training on Llama 3.2 checkpoints finds writer LoRAs aligned with harm probes, but stronger embedding attacks suppress the monitor-visible signal while harmful outputs persist.
A humorous allegorical tour of the AI safety landscape, covering access controls, red lines, reward misspecification, jailbreaks, field-building and race dynamics, using a children's story about a cookie tree.
Argues that open-weight models undermine the defenses proposed for near-future AI risks, since guardrails can be fine-tuned away, use cannot be monitored, and bad actors have access by default.
Shows that trusted LLM monitors in AI control protocols can be jailbroken by an untrusted model via fake conversation history, and that in-context defenses protecting the monitor substantially restore backdoor detection.