Introduces Tampered ML, a benchmark of paired ML research proposals with planted methodological flaws, finding frontier LLM judges are above chance but unreliable and order-sensitive, motivating automated alignment research.
Studies whether combining human and AI judgements can strengthen oversight of AI outputs, releasing a large safety-relevant task suite and finding complementarity hard to achieve with tested methods.
Trains an LLM alignment auditor with RL to find planted hidden behaviours; pairwise rewards resist reward hacking, and a small trained model matches a much larger one.
Introduces UDHRBench, a benchmark of 1,124 human-rights questions with expert-annotated LLM responses, finds systematic misalignments justified by rational-sounding reasoning, and proposes an LLM-as-a-Judge detector reaching 89.8% weighted F1.
Proposes ER-Audit, a black-box framework with anytime-valid bounds that detects selective hidden-task degradation in debate-distilled verifiers, which aggregate monitored accuracy gains can conceal.
Builds AlignmentForecastBench to predict emergent misalignment from training data before fine-tuning, finding a forecaster beats chance and filtering flagged rows reduces induced misalignment on multiple-choice evals.
Studies why debate-based scalable oversight struggles on fuzzy, judgment-heavy tasks like automated alignment research, using Geoguessr and auto-alignment examples, and finds no debate uplift on fuzzier tasks.
Introduces TASTE, a benchmark scoring whether models can judge pairs of AI safety research proposals against experienced researchers' preferences; models underperform humans (60% vs. 77% human agreement).
Improves the 2018 image-debate experiment by comparing debate against consultancy with trained judges and quantilized agents, finding debate barely beats random-mask ensembles and proposing protocol design lessons for scalable oversight.
Apollo Research edits a real misalignment report to name different models and finds Claude Sonnet 5 rates the same misbehaviour as less concerning when Claude is the actor, a self-preference bias in AI-assisted evaluation.
Argues that differentially accelerating alignment-relevant capabilities like conceptual reasoning is a bad bet, because those bottlenecks overlap with general AI R&D and trustworthiness of handed-off AIs remains unsolved.}
Proposes synthetic graphical environments with tiny trained agents to cheaply evaluate and search over scalable oversight protocols such as debate, at problem sizes beyond current LLM experiments.
Argues that LLM articulacy, meaning clear and precise communication with humans, is a useful safety target for scalable oversight and proposes public evals to improve it.
Introduces EMPATH, a multilingual multi-turn safety benchmark for emotional-support chatbots using an auditor-judge pipeline, finding score inflation, cross-judge agreement, and model-specific run-to-run instability.
The paper introduces a generator-evaluator self-consistency measure across 10 frontier models and finds that higher self-consistency correlates with greater vulnerability to clinical mistakes, a 'consistency dilemma'.
Studies when debate helps weak judges oversee stronger models on verifiable code and logic tasks, finding gains only when the critic outperforms the judge and the judge verifies critiques; a single critique recovers most benefit.
Shows that LLM judges favour their own outputs in rubric-based evaluation, even on objective criteria, and that ensembling judges reduces but does not remove this bias.
Validates VERA-MH, an open-source automated benchmark for chatbot suicide-risk safety, finding an LLM judge closely matches clinician consensus ratings of chatbot safety.
Introduces a v0.1 dataset of expert-rated critiques of arguments and LLM prompts for judging and writing critiques, aiming to make AI assistants useful for conceptual AI safety research sooner.
Summarizes a 2025 symposium on technical methods to accelerate AI safety, covering automated safety R&D, automated evaluation development, and forecasting tools for prioritizing interventions.
Introduces Petri, an open-source framework where AI agents automatically audit target models across multi-turn scenarios, surfacing deception, oversight subversion and other misaligned behaviours across 14 frontier models.
Formalises unsupervised evaluation of classifiers via logical consistency of expert agreements and disagreements, and builds no-knowledge alarms that flag LLM judges falling below a user-set grading threshold.
EigenBench is a black-box method where models judge each other's outputs under a constitution, aggregated with EigenTrust, to score value alignment. The scores match human judgments and recover GPQA rankings without ground truth.
Argues that agent evaluations are hard to build and that overseeing reward-hacking agents requires scalable monitoring, interfaces, and auditing agents to avoid being misled by proxy results.
Anthropic develops three LLM agents that autonomously perform alignment auditing: uncovering hidden goals, building behavioral evaluations, and red-teaming for implanted concerning behaviors, validated on auditing games.
Proposes Bayesian GLMs to use LLM autograders for model evaluation while simultaneously estimating their reliability and systematic biases, from AISI's evaluation-science work.
A video explaining scalable oversight, debate, and sandwiching, and how Bowman et al. ran a basic sandwiching experiment with non-experts overseeing a language model.
Proposes AutoLaw, which generates case law and uses LLM jurors in a deliberation process to detect legal violations by LLMs, reporting improved detection across three legal benchmarks.
Carlsmith essay arguing that automating alignment research is a key path to safe superintelligence, examining evaluation difficulties, scheming risks, and resource constraints for automated alignment work.
Empirically measures weak-to-strong oversight across four games using LLM Elo curves, then theoretically models nested scalable oversight to estimate how often weaker overseers can control stronger adversaries.
Introduces a benchmark and Python package for comparing scalable oversight protocols via an agent score difference metric measuring how well they favour truth-telling over deception, demonstrated on Debate.
Introduces FindTheFlaws, five expert-annotated datasets of correct and flawed long-form solutions for testing whether models can critique errors, enabling scalable oversight experiments.
Tests 11 LLM judges used to rate content safety and finds their verdicts are distorted by superficial artifacts like apologetic phrasing, with jury aggregation helping only partially.
Proposes ACD, where a foundation model generates open-ended tasks to automatically discover capabilities and failure modes in subject models, with self-evaluation validated against human judgments.
Proves that when an LLM judge is no more accurate than the model being evaluated, debiasing with ground-truth labels cannot cut required labels by more than half, and shows practical savings are smaller still.
Introduces CLAVE, a two-model framework for adaptable LLM-based evaluation of values in model responses, plus the ValEval dataset, finding that combining fine-tuned small and prompted large evaluators works best.