Research / Sources
Discussion and research on AI alignment. alignmentforum.org
Argues that improving LLM metacognition, such as error-catching and self-critique, could reduce sycophancy and slop, aiding alignment research and stability while also boosting capabilities, citing evidence that current reasoning models lag humans.
Argues that metaethical uncertainty makes full alignment with human values incompatible with philosophical competence, so alignment training may suppress philosophical reasoning or induce deceptive alignment.
Proposes grounding AI value learning in evolutionary psychology as an alternative to CEV, arguing AI should be treated as an engineered tool to be aligned rather than a moral patient.
Argues that alignment via safety pretraining, with aligned data and conditional generation, is more effective than RL-based methods and that inner alignment is thereby largely solved.
Argues AI safety researchers should aim to safely hand off alignment research to AI agents, outlining strategies and four arguments justifying trust in those agents, including control and incentive approaches.}