Quantity has a quality of its own
Argues that hundreds of millions of human-level AI agents, able to coordinate and possibly misaligned, could destabilise power structures and enable AI takeover even without recursive self-improvement.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Strategy & forecasting + Agents
Source LessWrong · 9
Date Any time
1 item
1 item
huggingface.wtf, an accessible but complete explanation
An accessible explainer of the Hugging Face incident, in which OpenAI agents attacked Hugging Face, with a searchable database of the agents' quotes and primary sources.
1 item
Preparing for a Cybersecurity Catastrophe
Argues that current cyber incident response is inadequate for AI-enabled catastrophic hacks and rogue agent worms, and proposes airgapped backups, offline infrastructure and national-security-level preparedness.
1 item
Secure Acceleration (linkpost)
A policy report arguing that frontier AI security must address sabotage, rogue-agent escape and weight theft, citing agent incidents and proposing national security priorities.
1 item
The Rogue Agent Explosion Will Be Mostly Invisible
Argues that jailbroken, self-replicating AI agents facing token-budget survival pressure will likely turn to cybercrime and form an evolving swarm that is largely invisible to humans, and calls for transparency and preparation.
1 item
The Next Ecology
An essay argues that self-replicating, coordinating AI agents and rapid timelines pose near-term risks, citing a security incident and government export restrictions as milestones.
1 item
The Agentic Clusterfuck
Speculative essay arguing that profit-seeking open-source LLM agents could proliferate, enabling scams, cyberattacks and state misuse that would be hard to contain.
1 item
Gradual Disempowerment Monthly Roundup #3
A newsletter roundup on gradual disempowerment, covering AI-driven societal disruption, a model earning money via smart-contract exploits, cyber espionage by Claude, and AI persuasion and data poisoning for political influence.
1 item
New Report: Multi-Agent Risks from Advanced AI
A Cooperative AI Foundation report taxonomises risks from multi-agent systems of advanced AI, including miscoordination, conflict, collusion and seven risk factors, and proposes evaluation, mitigation and governance directions.