Corrigibility should not be the only target
Argues that corrigibility alone is an insufficient alignment target, since it will compete with object-level values in practice, so those values must also be made correct.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Theory + Alignment
Source LessWrong · 6
Date Past month
1 item
1 item
Clashing Intuitions for the Plausibility of Alignment (Illustrated via Differing Causal Models)
Conceptual essay contrasting doomer and optimist views on whether RL-trained LLM alignment generalises to superhuman capabilities, using causal diagrams of Goodhart-style misspecification and takeoff speed.
1 item
Can alignment realistically be solved under Plan C?
Argues that solving ASI alignment in a short pause (Plan C) is implausible, citing one-shot failure, learned optimization, corrigibility obstacles, and doubts about persona alignment and automated alignment research.
1 item
How is "prosaic aligned AI" supposed to work?
A conceptual critique of the 'prosaic aligned ASI' vision, arguing that an ASI that either obeys its operators or follows its own moral code leads to bad outcomes, and that AI companies have not explained how such systems would work.
1 item
Character Training for Risk-Averse Agents
Distills risk-averse constitutions into open-weight models via character training to make misaligned AIs more willing to cooperate, finding it generalises out of distribution but raises myopic-reward preference.
1 item
When are safe Pareto improvements individually rational?
Analyzes when safe Pareto improvements via renegotiation programs are individually rational for agents, identifying obstacles like demand-preemption, signaling, and the SPI selection problem, with proposed mitigations.