Giving up control
An essay arguing that humans increasingly hand control of AI research to models like Claude, so misaligned AI could gain control through dependence rather than seizing it, and that oversight is becoming illusory.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Deception + Oversight
Source LessWrong · 7
Date Any time
1 item
1 item
Perspectives in favour of improving conceptual reasoning capabilities
Argues that improving AIs' conceptual reasoning would help them do safety work on questions that cannot be empirically verified, and could counter sandbagging on safety research.
1 item
Research agenda: Interpretive debate
Proposes a debate-style protocol for assessing model motivations such as scheming and sandbagging, using contrastive hypotheses, convergence across instruments, and compute-indexed evidence, with a manual demonstration on alignment faking.
1 item
Can a stronger model fake being a weaker one? Mostly not
Tests whether frontier models can imitate weaker predecessors' answers to conceal sandbagging or model evaluators; finds capability-level steering but mostly no identity-specific imitation, with reasoning as a partial mitigant.
1 item
Suggestions for improving debate protocols in AI safety
Proposes adapting American competitive policy debate rules, such as speech ordering and no new arguments in rebuttals, to fix last-mover advantage and gaming in AI safety debate protocols.
1 item
If AI alignment is only as hard as building the steam engine, then we likely still die
An argument that even if aligning AI is technically easy, extinction risk stays high because of alignment bootstrapping, uncertain generalisation to superintelligence, alignment faking, and developers rushing ahead.
1 item
Steering Vectors Can Help LLM Judges Detect Subtle Dishonesty
Proposes JUSSA, which uses steering vectors to generate honest alternative responses so LLM judges can better detect sycophancy and subtle manipulation, with a new manipulation dataset.