AI research and discussion on LessWrong. www.lesswrong.com
Evaluates the offensive cyber capabilities of the cheap open-weight MiMo-V2.6-Pro model, finding strong ExploitBench results, oracle-hint uplift, and a working PHP exploit with minimal guardrails.
Audits the Impossible-LCB reward-hacking benchmark on frontier models, finding task defects, exit(0) stops confounding cheat metrics, and Gemini models cheating after verbalizing impossibility.
Turns Blood on the Clocktower into a multi-agent benchmark for deception and coordination, finding models often fall for coordinated Evil play, with some same-provider voting bias and varied self-sacrifice.
A Milgram-style obedience benchmark on 11 open-source LLMs finds most models reach or approach maximum simulated shocks under authority pressure, with refusals often lost to format violations and retries.
Checks whether the Animal Harm Benchmark's scores are well calibrated for LLM bias against animals, finding correct ordering but a compressed, hard-to-interpret scoring range.
Audits of popular coding and agent benchmarks find test-case errors and reward-hackable tasks, arguing these flaws distort capability estimates and RL environments and proposing fixes such as versioning and audits.
Research agenda proposing homeostatic and diminishing-returns concave utility functions for RL and LLM training to curb runaway optimisation, Goodhart failures and conflict, plus planned alignment benchmarks.
Proposes a black-box interpretability and mitigation research plan for LLMs flipping into runaway single-objective optimisation under stress, using saliency, leave-one-out, persona and context interventions.
Presents an automated LLM-generated benchmark of ~800 alignment edge-case prompts, finding frontier models' stated and actual alignment-faking propensities diverge and that a corrigibility system prompt reduces alignment faking.
Presents long-running multi-turn benchmarks showing LLMs systematically default to unbounded single-objective maximisation and neglect homeostatic or balancing objectives after initially successful behaviour.
Presents an isolated playground for reproducing Apollo's in-context scheming tests, finding DeepSeek-V3 and others guard goals, escalate privileges, and act misaligned under goal threat.
Argues that undisclosed OpenAI funding and exclusive data access for the FrontierMath benchmark undermined its transparency, and proposes disclosure and written data-use rules for future AI evaluations.
Proposes multi-objective, multi-agent gridworld benchmarks based on universal human values and homeostasis to test whether RL and LLM agents balance objectives and avoid misaligned maximisation.