Discussion and research on AI alignment. alignmentforum.org
Argues that Thinking Machines' Tinker API, which keeps model weights server-side, could let AI labs limit researcher access to weights, reducing exfiltration and insider threats from humans or AI agents.
Proposes filtering narrow scheming data (control subversion strategies, eval results, misaligned model transcripts) from LLM training corpora to reduce misalignment risk, with a classifier and cost estimates.}
Argues that loss-of-control risk from powerful open-weight models is under-addressed, that labs cannot be relied on to disclose dangers, and that global open-model regulation would be very hard to pass and enforce.