Research preprints from the scientific community. arxiv.org
Proposes LARA, a Lagrangian dual-based reward augmentation for inference-time alignment of frozen LLMs under safety cost constraints, improving the helpfulness-harmlessness tradeoff with Best-of-N.
Position paper arguing that alignment techniques are dual-use and could be misused by malicious actors for censorship and manipulation, and proposing mitigation strategies.
Argues that misaligned AGI and aligned-AGI misuse pose catastrophic risks in tension, that many current alignment techniques may increase misuse risk, and that governance is essential.
A broad survey of LLM safety across the full lifecycle, covering data, pre-training, post-training, deployment and commercialization, drawing on over 800 papers.
Shows that safety fine-tuning on Llama 2-Chat 13B can be undone cheaply, under $200, while keeping general capabilities, so released weights make safeguards ineffective.