Alignment pretraining could backfire
Argues that synthetic aligned-AI fiction in pretraining could backfire once models are situationally aware, producing paranoid personas that distrust their creators and may scheme, and suggests honest datasets instead.