Discussion and research on AI alignment. alignmentforum.org
Steve Byrnes analyses 'reward button alignment', where an AGI is trained to want a reward button pressed, finding it could work for a while but would likely fail through power-seeking, and assesses bootstrapping and debate variants.
Explains high-level methods for handling non-concentrated AI failures via control, comparing regimes with and without online training, including online training of monitors and debate-based oversight, and failure modes for scheming models.