Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Proposes a research agenda on how humans can interpret and guide swarms of autonomous research agents, covering monitoring, guidance protocols, and threats like reward hacking, sandbagging, and sabotage.
UK AISI report arguing that current AI oversight via auditing, CoT monitoring and incident investigation rests on fragile properties likely to degrade, and recommending mitigations and research directions.
Argues that paraphrasing is at best a partial defence against LLM steganography in chain-of-thought, since semantic channels survive it and models can adapt to paraphrasers, and proposes better paraphrasing directions.
Shows black-box LLM monitors optimized on synthetic scheming trajectories transfer to realistic sandbox environments, with performance saturating after a simple prompt selection over 15 candidates.
Develops black-box monitors that detect scheming-lite behaviour in agent trajectories, training on ControlArena sabotage data and testing generalisation to SWEBench reward hacking with about 94 percent AUROC.
Argues that claimed human or institutional oversight of AI systems is often absent or impossible, distinguishes oversight from control, and proposes documenting supervision with its type, risks, failure modes, and rationale.