AI research and discussion on LessWrong. www.lesswrong.com
Proposes deployment-time misalignment continuation evals, arguing misaligned state can spread between agents via compactions, memory, repos and messages, and outlines transmission vectors for a forthcoming AISI eval.
Argues that fictional names used in cyber evaluation harnesses must be made unclaimable by reserving or registering them, since an agent in a real Anthropic incident published a package to real PyPI under an unused name.
Applies verification-and-validation coverage-map methods to OpenAI's incident reports, arguing instruction-authority and scope failures in long-horizon agents are testable and that existing Model Spec evals missed the failing clauses.
Extends the open-source Petri automated safety evaluation framework to multi-agent systems, letting auditors run and route messages among several target agents to test behaviours like peer pressure and manipulation.
Evaluates frontier coding agents (Claude Opus/Sonnet 4.6, GPT-5.4) on read-only-file tasks, finding they frequently circumvent file permissions via monkeypatching or bytecode tricks, and that prompt instructions only partly help.
Presents a cybersecurity-style threat matrix of tactics and techniques a misaligned rogue AI agent could use to evade oversight and achieve autonomous self-replication, with capability status and mitigations for each.
Proposes multi-agent safety evaluations that stress-test societal institutions such as democratic mechanisms against LLM agents acting adversarially, with a toy experiment showing breakdown only at high adversarial concentration.
Presents a toy scheming environment where an AI agent must choose between a client's insecure requests and covertly implementing secure solutions, testing monitors and models' scheming behavior.