Local Factor Graph Debate
Proves that the error of unstructured debate between two agents decays as a power law in debate length for weakly coupled factor graphs, using Ising-model analogies to separate local and nonlocal argumentation regimes.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Theory + Oversight
Source LessWrong · 13
Date Any time
1 item
1 item
Core Assumptions of ELK
Analyses the core assumptions behind ARC's Eliciting Latent Knowledge problem, arguing that solving it requires a large intersection between human and predictor ontologies and critiquing proposed solutions.
1 item
The Open Problems of the AI Alignment Field and their Cruxes
Maps open AI alignment problems, their cruxes and dependencies, and formalizes some of them, including debate and corrigibility, in Lean to identify gaps and collaboration opportunities.
1 item
What comes with cheap math?
Develops a formal decision-theoretic criterion for when humans can justifiably trust AI, modelling trust between logical inductors and linking it to misalignment and learning normativity, with Lean-verified results and noted limitations.
1 item
theory uplift differentially benefits safety & is underleveraged
Argues that AI-powered mathematical verification could give safety guarantees before automated AI R&D matures, and that funding and infrastructure for turning compute into safety-relevant proofs is severely underleveraged.
1 item
Automated Alignment is Harder Than You Think
Argues that using AI agents to automate alignment research could yield misleadingly optimistic safety assessments, because hard-to-supervise fuzzy tasks and correlated errors resist error correction and existing oversight methods.
1 item
Automated AI R&D and AI Alignment
A philosophy-of-science essay on automating alignment research, arguing that AI can help with verification of internal consistency but validation of whether a system's objectives are right is much riskier to hand off.
1 item
Corrigibility Scales To Value Alignment
Argues that corrigibility, as defined by Max Harms, scales naturally into an approximately value-aligned superintelligence and that alternatives do not reduce existential risk, while acknowledging significant uncertainties.
1 item
IABIED: Paradigm Confusion and Overconfidence
A critical review of the book If Anyone Builds It, Everyone Dies, arguing its alignment paradigm, dismissal of superalignment, takeoff-speed and corrigibility claims are overconfident.
1 item
[Question] What the discontinuity is, if not FOOM?
A LessWrong question post surveys critics' responses to Yudkowsky and Soares on whether a sharp discontinuity to superintelligence is needed for alignment to be infeasible under gradual progress.
1 item
Alignment Proposal: Adversarially Robust Augmentation and Distillation
Proposes ARAD, an alignment plan where a principal safely consults slightly smarter advisors and distills the pair iteratively, using agent-foundations ideas such as cryptographic pessimism; the author flags unresolved flaws.
1 item
The prospect of accelerated AI safety progress, including philosophical progress
Essay arguing that safe superintelligence may require decades of philosophical progress, possibly accelerated by human-AI collaboration, and that loss of human control is the central risk on the current path.
1 item
Topological Debate Framework
Proposes topological debate, where two AI agents argue over which simulation resolutions or world models determine a safety property, linking AI safety via debate to guaranteed safe AI.