Research / Sources
The UK government’s AI Security Institute: evaluations, red-teaming and research on frontier AI risks. aisi.gov.uk
Argues alignment is largely a human-supervision problem and proposes a cognitive and social science research programme on five bottlenecks: biased judgement, plural values, limited attention, counterfactual reasoning, and verification.
Argues that using AI agents to automate alignment research could yield confident but misleading safety assessments, because fuzzy, hard-to-supervise tasks resist human oversight, and calls for training agents to handle them.
Proposes Bayesian GLMs to use LLM autograders for model evaluation while simultaneously estimating their reliability and systematic biases, from AISI's evaluation-science work.
Proposes a Bayesian GLM framework to measure LLM autograder biases and reliability, quantifying scoring differences and uncertainty in LLM-as-a-judge evaluations.
Proposes a recursive AI debate protocol that prevents dishonest debaters from winning via obfuscated arguments, so honest debaters need only comparable computational effort to prevail.
Sketches an alignment safety case based on AI debate, arguing debate-trained agents stay honest in deployment to prevent sabotage by an AI R&D agent, and lists open research problems.