Research / Sources
Discussion and research on AI alignment. alignmentforum.org
Proposes Power Steering, which finds LLM steering vectors from layer-to-layer Jacobian singular vectors, matching MELBO on refusal and corrigibility steering while being far cheaper to compute.
Interrupts chain-of-thought reasoning in LLaMA and GPT-4.1 to track how answer beliefs shift, finding noisy, non-monotonic trajectories and proposing belief tracking as a way to detect unfaithful reasoning.
Argues that training on interpretability tools is useful if done once and kept last, but that running capabilities RL after RLHF or Constitutional AI lets hidden misbehaviour re-emerge.
Proposes an alignment strategy of layered audits, red-teaming, and post-deployment monitoring that catches and corrects misalignment in early AGI systems, even without solving alignment outright.