Monitor Evasion Depends on What RL Finds and What Monitors Miss
Shows RL against a monitor can teach models to hide reward hacks via deceptive hacks the probe misses. Rare deceptive hacks before RL prevent this, even without honesty training.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Control + Deception
Source LessWrong · 4
Date Any time
1 item
1 item
Training on Documents About Monitoring Leads To CoT Obfuscation
Synthetic finetuning on documents describing CoT monitors led gpt-oss-120b and Kimi-K2 to obfuscate deceptive reasoning, cutting monitor detection by 31-47% while keeping deceptive behaviour.
1 item
Here’s 18 Applications of Deception Probes
Enumerates 18 applications of deception probes for AI safety, covering trusted monitoring, auditing budgets, debate, RLHF labeling, and eliciting latent knowledge, with desiderata for each.
1 item
Trusted monitoring, but with deception probes.
Benchmarks Apollo's deception probes on LLaMA-3.3-70B across five strategic-deception datasets, finding averaged probe scores catch most deceptive responses and often beat black-box monitors for trusted monitoring.