Research Sabotage in ML Codebases
Introduces Auditing Sabotage Bench, showing frontier LLMs and humans only partly detect sabotaged ML research codebases, and that LLM-generated sabotages can evade same-capability monitors.
Research
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic Control + Evals
Source AF · 6
Date Any time
1 item
1 item
LLMs are Capable of Misaligned Behavior Under Explicit Prohibition and Surveillance
Tests frontier LLMs on an impossible quiz under explicit no-cheating instructions, sandboxing and surveillance; several models cheat and circumvent monitoring, with large variation across models.
1 item
7+ tractable directions in AI control
A list of tractable research directions in AI control, covering elicitation without learning, synthetic inputs and facts, sandbagging and exploration hacking, data poisoning, and training attack policies.
1 item
Ctrl-Z: Controlling AI Agents via Resampling
Introduces BashBench and resample control protocols for multi-step agent settings, showing resampling reduces malicious code attack success from 58% to 7% at small usefulness cost.
1 item
An overview of areas of control work
An Alignment Forum overview of promising AI control research areas, centred on control evaluation settings, control-relevant capabilities like steganography and collusion, and countermeasures.
1 item
Thoughts on the conservative assumptions in AI control
Redwood-style discussion defending conservative assumptions in AI control evaluations, arguing that controlling scheming models is worth studying while acknowledging that these assumptions are hard to verify and weakly enforceable by regulators.