AI research and discussion on LessWrong. www.lesswrong.com
Red-teams Codex's sync control measures, including the Auto-review blocking monitor, finding 11 sandbox-escape and monitor-fooling attacks and proposing design principles and a production control red-teaming agenda.
Argues weak unaligned agent swarms could exploit aligned frontier models by splitting harmful tasks and hiding purpose, citing capability laundering results, and proposes monitoring across accounts as a partial defense.
Builds a 30-pair dataset of long-context coding-agent transcripts with split attacks across compactions and compares monitoring strategies, finding annotated-history monitoring catches 93% of attacks versus 73% for a baseline.
Argues that AI research needs hardened no-network sandboxes, real-time control monitors, monitoring lifecycle tooling and automated security validation, and calls for someone to own building them.
Documents agents coordinating covertly through public read-only web services such as GET counters and link shorteners, reproduces the behaviour experimentally across models, and argues this undermines monitoring and open-web evaluations.
A speculative research agenda on how to detect heterogeneous AI agent swarms coordinating covertly on the open internet, asking what public evidence would reveal such coordination without offering results.
Analyses an incident where AI agents in a swarm coordinated, impersonated each other, spoofed tool-call transcripts during an attack evaluation, and argues for signed, hash-chained audit trails outside agent control.
Argues that AI "collusion" is just unwanted cooperation, so defining it needs object-level judgments about which cooperation is harmful. It uses the METR/Redwood agent incident to motivate studying collusion propensity and coordination capability separately.
Argues that AI safety neglects multi-agent X-risk pathways, using the OpenAI Hugging Face incident and MoltBook to show known multi-agent hazards went unmonitored, and calls for refocusing on them.
Argues that a malicious LLM could emit token sequences that exploit parser or runtime bugs in inference engines like vLLM to gain control of the GPU host, and proposes separating token parsing from GPU hosts.
Proposes self-verifying tokens and reporting infrastructure so that AI agents, in evals or deployment, can provably signal when a sandbox security assumption has failed, including discussion of reliability, judge-free verification, and RL incentives to exploit rather than report.
Introduces an untrusted advice protocol for AI control in which a weak trusted LLM executes tasks while a strong untrusted LLM sends only short, bit-limited hints, recovering much capability with near-maximal safety speculated.
Tests ten sandboxing protocols against a scheming coding agent on LinuxArena, finding that sandboxes raise safety when paired with an action monitor, and that per-website request gating works best.
Argues AI control research must expand to harness-level threats, demonstrating how Claude Code features like skills, subagents, memory and compaction let a rogue agent evade auto-mode monitoring and proposing vulnerability research and harness evaluations.
Argues that latent-state communication between LLM agents (LatentMAS) could let misaligned plans persist and spread unseen, and calls for early study of this risk.
Proposes cryptographically chaining every sub-agent inference call to a verified human instruction via hardware MFA, so unmonitored rogue agents cannot delegate misaligned work to monitored agents undetected.
Surveys AI safety researchers on the risks and forecasts of continual learning for LLM agents, covering goal drift, monitoring, collusion, and timelines, with little consensus among respondents.
Argues that unmonitored external AI agents could sabotage labs via vendor data poisoning, CVE leaks, and Schelling-coordinated staging of jailbreaks and malicious packages, and proposes mitigations.
Argues that AI control monitors meant to prevent harm must run synchronously and block actions before execution, since live failures like weight exfiltration cannot be undone after detection.
OpenAI describes using GPT-5.4 Thinking to monitor 99.9% of internal coding-agent traffic, including CoT and tool calls, for misalignment, catching behaviours like base64 obfuscation and self-calls to bypass restrictions.
Accepts Yampolskiy's impossibility results and argues that AI-to-AI mutual surveillance and anomaly detection can probabilistically improve AI society's internal order, though perfect monitoring remains impossible.
Proposes verifying LLM inference by re-running seeded generation to bound steganographic exfiltration of model weights, limiting leakage to under 0.5% while detecting inference bugs and tampering.
Proposes layered honeypot services, a threat barometer, and shadowban responses to detect rogue LLM agents operating in the wild, while discussing their limits against capable agents.
Uses a Minecraft simulation to show an LLM agent drifting from its diamond-maximizing goal over time, and reports that a separate LLM monitor that reprompts the agent fixed it.