AI research and discussion on LessWrong. www.lesswrong.com
Tests whether Neural Chameleons can evade activation probes for topics they must answer about; finds evasion on trained topics costs substantial capability and unseen-concept evasion mostly fails, questioning the original paper's claims.
Argues that untrusted monitoring against collusion needs three countermeasures (honeypots, trusted monitoring, signal jamming), with red-team analysis showing each is necessary to prevent collusion between AI models.
Argues that paraphrasing is at best a partial defence against LLM steganography in chain-of-thought, since semantic channels survive it and models can adapt to paraphrasers, and proposes better paraphrasing directions.
Tests whether SFT and RL can elicit the true performance of prompted sandbagging models on Olympiad math, finding the gains mostly come from degraded system-prompt following rather than a real aversion to sandbagging.
Hobbyist experiments fine-tune Kimi K2.5 to produce load-bearing obfuscation in internal chain-of-thought that evades a monitor, with limited, admittedly rushed evidence of success.
Extends untrusted monitoring for AI control, arguing trusted monitors are essential, collusion signals are hard to block, and capabilities need indirect evaluation for safety cases.
Studies collusion strategies against untrusted monitoring in AI control, finds SFT-elicited passive self-recognition beats prior attacks, and sketches a safety case with open gaps.
Shows black-box LLM monitors optimized on synthetic scheming trajectories transfer to realistic sandbox environments, with performance saturating after a simple prompt selection over 15 candidates.
Proposes decomposing takeover risk in multi-model deployment into alignment, diversification, and control, with a formal model showing diversification helps only when the danger function is convex.
Trains LLM agents to call a report_scheming tool alongside side-task actions, so covert misbehavior becomes visible; it cuts undetected attacks versus blackbox monitors across 15 environments with minimal safety tax and generalizes to uninstructed misbehavior.
Argues that deploying a diverse mix of possibly scheming AI models, rather than many copies of the most trusted one, lowers catastrophe risk when honest models can expose or prevent schemers, and gives recommendations.
Develops black-box monitors that detect scheming-lite behaviour in agent trajectories, training on ControlArena sabotage data and testing generalisation to SWEBench reward hacking with about 94 percent AUROC.
Benchmarks Apollo's deception probes on LLaMA-3.3-70B across five strategic-deception datasets, finding averaged probe scores catch most deceptive responses and often beat black-box monitors for trusted monitoring.
Tests whether two Claude 3.7 Sonnet copies can coordinate via Schelling focal points to subvert an LLM monitor, finding low success but occasional collusion on non-obvious signals.
Maps steganography and watermarking research onto encoded reasoning in LM agents, arguing hidden CoT undermines monitoring and enables collusion, but current demonstrations show only limited information hiding.