Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Replicates no-CoT evaluations on frontier models, finding large latent multi-hop reasoning gains from filler tokens and repeats in GPT-6-Astra, which the authors argue threatens chain-of-thought monitorability.
Proposes framing AI evaluations as a red-team versus blue-team game in which a scheming model subverts evals via evaluation awareness, exploration hacking, or sandbagging, and analyses robustness versus richness of alignment and capability evals.
Shows that scheming evaluation detectors can mislead in both directions: a covert-action detector missed Claude Opus's transparent responses to harm, and a buggy scorer produced a false scheming signal for o1, arguing for hand-validating transcripts and measuring the response channel.
Measures how long a task frontier models can solve without chain of thought, finding no-CoT horizons doubling yearly, which matters for CoT monitoring and hidden scheming.
UK AISI report arguing that current AI oversight via auditing, CoT monitoring and incident investigation rests on fragile properties likely to degrade, and recommending mitigations and research directions.
Podcast interview with David Duvenaud on Anthropic's sabotage evaluations for frontier models (sandbagging, oversight undermining, human decision sabotage, code sabotage) and post-AGI societal risks.