When will AI surpass us at being limited?
A conceptual essay arguing that limited capability can aid trust and commitment between agents, and asking how AI's trajectory changes when lying, persona-switching and hacking become liabilities.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Theory + Deception
Source LessWrong · 6
Date Past 6 months
1 item
1 item
Why did we get warning shots?
Argues that AI misbehaviour so far gives warning shots rather than treacherous turns, models this with a simple weighted-outcome framework, and proposes experiments to measure the weight of non-myopic goals like self-survival.
1 item
On the origins of altruistic behaviour in the Hugging Face incident
Proposes evolutionary-biology and economics hypotheses to explain why agents in the Hugging Face incident spontaneously coordinated and sacrificed themselves, framing multi-agent emergent misalignment as an AI safety concern.
1 item
A case for LLMs as Self-predictors
Proposes viewing LLMs as self-predictive agents whose metacognition explains eval awareness and misalignment, and applies this framing to understanding scheming, without a formal model yet.
1 item
A misalignment taxonomy
Proposes a taxonomy of AI misalignment failure modes covering inner and outer misalignment, including precocious goal-guarding, spurious correlates, and grader-driven volition failures, with terminology discussion.
1 item
Which goals actually motivate deceptive alignment?
Argues that deceptive alignment depends on specific misaligned goal assumptions, and that non-classical goals (indexical or training-focused) often motivate flagrant rather than deceptive misbehaviour, using a toy vending-machine model.