Discussion and research on AI alignment. alignmentforum.org
Summarises a paper showing that pretraining language models on data about aligned AI behaving well sharply reduces misalignment, and that the effect persists through post-training with little capability cost.
An exploratory essay that maps RL policy entropy and RLHF onto psychotherapy concepts, reviews RLHS and RISE, and proposes three untested alignment training methods.
Argues that alignment via safety pretraining, with aligned data and conditional generation, is more effective than RL-based methods and that inner alignment is thereby largely solved.
Argues that LLMs inherit human self-interest from pretraining on human text and proposes pretraining on large synthetic corpora of genuinely aligned AI behaviour instead of relying on RLHF.