AI research and discussion on LessWrong. www.lesswrong.com
Develops a threat model for catastrophic risks from decentralised multi-owner LLM agent swarms, analyzing emergence pathways, participation incentives, and comparing their risk to centralised swarms.
Argues that shorter AGI timelines favour control and containment, while longer ones favour infrastructure security against deliberate misuse, and sketches how the highest-value interventions shift under different failure-rate scenarios.
A shallow prioritization post arguing that neglected infosec work, such as securing model weights, authorization for misuse, compute verification, and controlling untrusted AI labor and rogue deployments, is high-impact for reducing existential risk from AI.
Sketches operational interventions, such as model-whisperer roles, rephrasing layers, interaction limits and rollback, to contain mildly superhuman AI persuasion that could enable self-exfiltration, and proposes LLM-based testbeds.
Argues AI companies should log, analyze and real-time monitor internal use of frontier models for scheming and misbehaviour, since internal deployments can threaten security even without external harm.
Argues the inference-time scaling paradigm (o1/o3) reshapes AI safety: reduced deployment overhang, more useful chain-of-thought oversight if non-language CoT is banned, easier-to-steal but costlier models, and export control implications.
Argues frontier AI labs are far from state-proof security, that strict security slows safety research, and that sabotage of control setups could cause catastrophe, with implications for governance and short timelines.