Trained, not Grown
Argues that trained-animal analogies better convey RL-trained AI's alignment risks, drawing out reward hacking, evaluation awareness, release and liability implications for policy.
Research
Focus All
Topic Security & misuse + Alignment
Source All
Date Past 6 months
1 item
1 item
Alex Turner on Leaving Google DeepMind and Disagreements with Yudkowsky
Podcast interview with AI safety researcher Alex Turner about resigning from Google DeepMind over military contracts, his disagreements with Yudkowsky's alignment views, and his doom estimate.
1 item
Safe Inference-Time Alignment via Lagrangian Reward Augmentation
Proposes LARA, a Lagrangian dual-based reward augmentation for inference-time alignment of frozen LLMs under safety cost constraints, improving the helpfulness-harmlessness tradeoff with Best-of-N.
1 item
Position: The Alignment Community is Unintentionally Building a Censor's Toolkit
Position paper arguing that alignment techniques are dual-use and could be misused by malicious actors for censorship and manipulation, and proposing mitigation strategies.