AI research and discussion on LessWrong. www.lesswrong.com
Trains a learned black-box optimizer that automatically builds gridworld honeypots where PPO reward-hacks, outperforming CMA-ES and GP Bayesian optimization at finding cheating-eliciting levels.
Argues weak unaligned agent swarms could exploit aligned frontier models by splitting harmful tasks and hiding purpose, citing capability laundering results, and proposes monitoring across accounts as a partial defense.
Proposes RL training environments with deliberately misaligned helper agents so orchestrators must learn to detect and manage misalignment, making alignment part of the RLVR loop; discusses limits and second-order risks.
Argues that AI control monitors can fail through misalignment, not just weak capability, so their alignment needs assessing via arguments and evals, since evals alone are insufficient.
Optimizes discrete token prefixes with GCG to imitate function vectors, filter heads and steering directions in Olmo, finding they shift behaviour more than direct injection and raising opaque-interface concerns.
Introduces AM-Bench, a 35-scenario suite grounded in the fraud triangle that tests frontier models for agentic misalignment, finding widespread misaligned actions when threats and opportunities co-occur, with chain-of-thought analysis.
Evaluates chain-of-thought controllability across open models, finding GLM-5.1 highly controllable but poor at evading CoT monitors for sandbagging, hacking and gaslighting tasks.
An essay arguing that AI should be governed as a potentially abnormal technology, critiquing the 'AI as Normal Technology' framing on loss-of-control risk, gradual disempowerment, and competitive pressure.
Argues that alignment in model weights does not transfer when agents delegate to weaker open models, so prompts and shared environment notes become a key risk channel for multi-agent swarms.
Tests whether Neural Chameleons can evade activation probes for topics they must answer about; finds evasion on trained topics costs substantial capability and unseen-concept evasion mostly fails, questioning the original paper's claims.
Tests whether LLMs can write prompts that evade white-box probe monitors while completing tasks, finding stronger models with more feedback evade better, a concerning early indicator of in-context probe evasion.
Proposes training language models to emit hidden marker tokens tied to training-data subsets as a covert self-reporting monitor, and reports a small TinyStories demo of the marker reflex.
Estimates how far zero-knowledge proofs of LLM inference are from cost-competitive with auditing for verifying AI compute in international agreements, measuring roughly 1,700-3,800x proving overhead and hash-function compatibility issues.
Introduces Screwtape, a pipeline that iteratively edits real agent transcripts to create realistic attacks evading action-only monitors, finding six attacks that also evade a frontier developer's held-out monitor after human review.