Alignment Forum

Discussion and research on AI alignment. alignmentforum.org

AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)

22 Dec 2025BlogTheory

AIXI with general utility functions: "Value under ignorance in UAI"

Cole Wyeth

Recasts AIXI's semimeasures as credal sets to define general utility functions, recovering standard value functions and obtaining a wider class with max-min optimal agents, motivated by safety but largely theoretical.

AIXIAI

7 Jan 2025BlogTheory, Deception

Responses to ~all criticisms of AIXI

Cole Wyeth

A long Alignment Forum post by a researcher defending AIXI as a useful model for agent foundations, responding point by point to standard criticisms such as embeddedness, anvil problems, code exposure, decision theory, tiling and malign priors.}

AIXIAI