Research / Sources
Discussion and research on AI alignment. alignmentforum.org
Google DeepMind and UK AISI present a safety case sketch for control monitoring of frontier agent deployments, comparing asynchronous, semi-synchronous and fully-synchronous monitors on oversight, latency and harm-prevention conditions.
Proposes filtering AI-misalignment discussion from pretraining data to reduce hyperstitional misalignment risk, weighing costs to safety research and deployment, and argues labs should weakly adopt it.
Argues that Thinking Machines' Tinker API, which keeps model weights server-side, could let AI labs limit researcher access to weights, reducing exfiltration and insider threats from humans or AI agents.
Proposes filtering narrow scheming data (control subversion strategies, eval results, misaligned model transcripts) from LLM training corpora to reduce misalignment risk, with a classifier and cost estimates.}
Argues LLM action monitors for misaligned agents can live in four places (agent scaffolds, cyber detection systems, LLM APIs, code review), comparing strengths, weaknesses and organisational ownership to prioritise deployment.
Argues AI safety research must plan for deploying known scheming models, analysing how catching escape attempts lets developers reduce risk via classifiers, policy training, and monitoring, and recommends a responsible disclosure and third-party deployment policy.
Prioritizes threats for AI control, arguing that undetected rogue internal deployments with escalated permissions are worse than self-exfiltration and should be the top priority to prevent or catch.
Argues that AI control may raise existential risk by shifting outcomes toward ineffective warning shots and takeover, since controlled incidents are unlikely to prompt serious action on alignment.
Sketches a structured safety case arguing an internally deployed LLM agent won't exfiltrate data, based on a red-team control evaluation of monitoring protocols and extrapolation to deployment.
Apollo Research staff argue AI labs should publish detailed short-timeline (2027) plans, proposing minimal requirements: secure model weights and monitoring of scheming AIs, with faithful chain-of-thought and monitoring as core defenses.