Discussion and research on AI alignment. alignmentforum.org
Argues that latent reasoning architectures like COCONUT and full-bandwidth transformers would undermine chain-of-thought monitorability, weakening a key oversight tool against misalignment and scheming.
Tests whether DiffusionGemma uses its vector-valued diffusion state for hidden computation, finding its chain-of-thought stays largely monitorable and that probes, steering and J-lens transfer from Gemma.
Transparency audit of DiffusionGemma finds its latent-style diffusion reasoning can be made variable-transparent via interpretable token bottlenecks, though algorithmic transparency remains lower than autoregressive models, with similar monitorability to Gemma.
Evaluates whether natural-language activation verbalizers can surface a model's internal reasoning in a single forward pass, finding reconstruction is weak and verbalizations often look confabulated.