Local Factor Graph Debate
Proves that the error of unstructured debate between two agents decays as a power law in debate length for weakly coupled factor graphs, using Ising-model analogies to separate local and nonlocal argumentation regimes.
Research / Sources
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic All
Date Any time
Quantilized debate and consultancy in image environments: protocol design lessons for scalable oversight experiments
Improves the 2018 image-debate experiment by comparing debate against consultancy with trained judges and quantilized agents, finding debate barely beats random-mask ensembles and proposing protocol design lessons for scalable oversight.
Synthetic Scalable Oversight
Proposes synthetic graphical environments with tiny trained agents to cheaply evaluate and search over scalable oversight protocols such as debate, at problem sizes beyond current LLM experiments.
Research update: RL on Debate Games shows Proposal Accuracy uplift alongside Judge Hacking
Reports RL training on AI debate games for math proposals, finding proposal accuracy improves but debaters learn to exploit the judge via spurious harsh critiques, a protocol-level failure.
Suggestions for improving debate protocols in AI safety
Proposes adapting American competitive policy debate rules, such as speech ordering and no new arguments in rebuttals, to fix last-mover advantage and gaming in AI safety debate protocols.
When does debate help a weak judge? Evidence from code, logic and math
Empirically tests non-adversarial AI debate for weak judges across code, logic and math, finding debate beats consultancy when the judge exceeds a capability threshold, and rebuttals add little.
ECL-pilled models write constitutions for ASI
Tests how steerable frontier LLMs are toward cosmic-host values when conditioned on in-context constitutions, finding Gemini most steerable, Anthropic and OpenAI models resistant, and suffering-prevention priors overriding cosmic reasoning.
Rejecting Violence as an AI Safety Strategy
Argues that violence against AI developers would increase existential risk from AI, reviewing how such tactics would backfire through force asymmetry, policy capture, securitization, and stigma.
Topological Debate Framework
Proposes topological debate, where two AI agents argue over which simulation resolutions or world models determine a safety property, linking AI safety via debate to guaranteed safe AI.