AI research and discussion on LessWrong. www.lesswrong.com
Argues that LLM advisors sitting as mediators between game players could steer outcomes toward a consistent lean, a quiet loss of human control, and proposes a multi-arm test to detect it.
Argues that model weight exfiltration is a less critical step in loss-of-control scenarios than often assumed, since models could take over developer organisations, and suggests reducing emphasis on exfiltration evals.
Uses item response theory to measure diminishing information returns from adding items to the Omni-MATH benchmark, finding the residual-correlation estimate is too noisy to predict benchmark size gains from small subsets.
Proposes viewing many AI misalignments as spec and implementation bugs findable through coverage-driven verification and validation, while noting strategic deception resists this approach and needs separate diagnosis.
Introduces 'entanglement' (information an AI has about its environment) and argues minimum entanglement lower-bounds the cost of credible AI evaluation environments, with implications for oversight.
UK AISI methodology post arguing that propensity research for misaligned LLM behaviour must model models' decision-making to test theoretical misalignment arguments, and proposing scalable, predictive methods to do so.
CLR's research agenda on safe Pareto improvements to prevent catastrophic AI bargaining conflict, covering SPI-incompatibility evals, conceptual conditions for SPI adoption, and research automation.
An informal probe of how major LLMs answer Newcomb, Sleeping Beauty and Smoking Lesion dilemmas, finding unanimous one-boxing and smoking, which the author links to alignment via decision theory.
Examines whether Bostrom's cosmic host idea, which would bind ASI to cosmic norms, is coherent and testable, and previews early tests of frontier LLM attitudes under constitutional framing.
An opinion essay arguing that current AI safety work, mainly behavioural testing and mechanistic interpretability, will not generalise to superintelligence, and that conceptual work on controlling outcome-steering systems is neglected.
Critiques Anthropic's Hot Mess paper, arguing its 'incoherence' metric is uninformative, its scaling claims are overstated, and its alignment conclusions don't follow from the experiments.
Argues that dangerous capabilities such as self-exfiltration, persuasion and AI research can emerge suddenly from gradual underlying progress once a threshold is crossed, using coding and chess analogies.
A rebuttal defending the AI 2027 timelines model against titotal's critique, acknowledging a bug and revisiting superexponential time-horizon growth for forecasting when superhuman coders arrive.
Formal analysis of minimal, agent-agnostic world models for sandboxing AI agents in safety evaluations, showing an efficiency-versus-interpretability trade-off and proposing belief-based alternatives.