AI research and discussion on LessWrong. www.lesswrong.com
A grantmaker explains which corrigibility research proposals they will not fund, including agent control layers, repeated misbehaviour tests, and copycat or off-target projects, and what they want instead.
Argues that AI control monitors can fail through misalignment, not just weak capability, so their alignment needs assessing via arguments and evals, since evals alone are insufficient.
Proposes simulated control evaluations using LLM environment simulators to test whether deployment monitors catch deliberately harmful agents, finding simulated results closely match real executed runs in LinuxArena.
Evaluates chain-of-thought controllability across open models, finding GLM-5.1 highly controllable but poor at evading CoT monitors for sandbagging, hacking and gaslighting tasks.
Evaluates single-forward-pass planning across model releases as an early-warning signal for evading chain-of-thought monitoring, finding Astra's no-CoT planning improved sharply and RL training can raise it.
Builds a 30-pair dataset of long-context coding-agent transcripts with split attacks across compactions and compares monitoring strategies, finding annotated-history monitoring catches 93% of attacks versus 73% for a baseline.
Tests whether LLMs can write prompts that evade white-box probe monitors while completing tasks, finding stronger models with more feedback evade better, a concerning early indicator of in-context probe evasion.
Shows LLMs can in-context edit activations to weakly evade latent monitors using only prior verdicts, and that amplification or a rank-1 LoRA organism evades held-out monitors, calling for activation-control evaluations.
Introduces Screwtape, a pipeline that iteratively edits real agent transcripts to create realistic attacks evading action-only monitors, finding six attacks that also evade a frontier developer's held-out monitor after human review.