No instrumental convergence without AI psychology
Argues that instrumental and success-conditioned convergence both depend on assumptions about AI psychology and plan-selection, so danger cannot be attributed to plan-space or reality alone.}
Research / Sources
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (577)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)AI Safety (3)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic All
Date Any time
The Bleeding Mind
Conceptual essay arguing LLM personas inherit human-text biases that leak information across character boundaries, causing hallucination, manipulation and spiritual-guru behaviour, and assessing self-other overlap alignment.
AXRP Episode 42 - Owain Evans on LLM Psychology
A podcast interview with Owain Evans covering LLM introspection experiments (Looking Inward), self-prediction, backdoors, and emergent misalignment, with safety motivations like honesty and detecting unintended goals.
Policy Entropy, Learning, and Alignment (Or Maybe Your LLM Needs Therapy)
An exploratory essay that maps RL policy entropy and RLHF onto psychotherapy concepts, reviews RLHS and RISE, and proposes three untested alignment training methods.
The Pando Problem: Rethinking AI Individuality
Conceptual post arguing that human-centric assumptions about AI individuality distort safety reasoning about scheming, goal preservation, and coordination among AI systems, with stories illustrating the risks.