Trained, not Grown
Argues that trained-animal analogies better convey RL-trained AI's alignment risks, drawing out reward hacking, evaluation awareness, release and liability implications for policy.
Research
Focus All
Topic Security & misuse + Alignment
Source All
Date Any time
1 item
1 item
Alex Turner on Leaving Google DeepMind and Disagreements with Yudkowsky
Podcast interview with AI safety researcher Alex Turner about resigning from Google DeepMind over military contracts, his disagreements with Yudkowsky's alignment views, and his doom estimate.
1 item
Safe Inference-Time Alignment via Lagrangian Reward Augmentation
Proposes LARA, a Lagrangian dual-based reward augmentation for inference-time alignment of frozen LLMs under safety cost constraints, improving the helpfulness-harmlessness tradeoff with Best-of-N.
1 item
Position: The Alignment Community is Unintentionally Building a Censor's Toolkit
Position paper arguing that alignment techniques are dual-use and could be misused by malicious actors for censorship and manipulation, and proposing mitigation strategies.
1 item
Dario Amodei – The Adolescence of Technology
Dario Amodei's essay maps civilizational risks from powerful AI, including autonomy, misuse, power seizure and economic disruption, and argues for judicious, limited regulation alongside voluntary company actions.
1 item
Misalignment or misuse? The AGI alignment tradeoff
Argues that misaligned AGI and aligned-AGI misuse pose catastrophic risks in tension, that many current alignment techniques may increase misuse risk, and that governance is essential.
1 item
A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
A broad survey of LLM safety across the full lifecycle, covering data, pre-training, post-training, deployment and commercialization, drawing on over 800 papers.
1 item
The ethics of advanced AI assistants
DeepMind's ethics foresight paper surveys ethical and societal risks of advanced AI assistants, covering value alignment, safety and misuse, economic and information impacts, and access.
1 item
BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B
Shows that safety fine-tuning on Llama 2-Chat 13B can be undone cheaply, under $200, while keeping general capabilities, so released weights make safeguards ineffective.