AI research and discussion on LessWrong. www.lesswrong.com
Builds AlignmentForecastBench to predict emergent misalignment from training data before fine-tuning, finding a forecaster beats chance and filtering flagged rows reduces induced misalignment on multiple-choice evals.
Studies why debate-based scalable oversight struggles on fuzzy, judgment-heavy tasks like automated alignment research, using Geoguessr and auto-alignment examples, and finds no debate uplift on fuzzier tasks.
Improves the 2018 image-debate experiment by comparing debate against consultancy with trained judges and quantilized agents, finding debate barely beats random-mask ensembles and proposing protocol design lessons for scalable oversight.
Apollo Research edits a real misalignment report to name different models and finds Claude Sonnet 5 rates the same misbehaviour as less concerning when Claude is the actor, a self-preference bias in AI-assisted evaluation.
Argues that differentially accelerating alignment-relevant capabilities like conceptual reasoning is a bad bet, because those bottlenecks overlap with general AI R&D and trustworthiness of handed-off AIs remains unsolved.}
Proposes synthetic graphical environments with tiny trained agents to cheaply evaluate and search over scalable oversight protocols such as debate, at problem sizes beyond current LLM experiments.
Argues that LLM articulacy, meaning clear and precise communication with humans, is a useful safety target for scalable oversight and proposes public evals to improve it.