An operationalization of opaque serial depth
Operationalizes NLS depth, a measure of unverbalized serial computation in models, proposing criteria for natural-language-rooted nodes to track CoT monitorability risks from architectural changes.
Discussion and research on AI alignment. alignmentforum.org
Operationalizes NLS depth, a measure of unverbalized serial computation in models, proposing criteria for natural-language-rooted nodes to track CoT monitorability risks from architectural changes.
Proposes that AI companies publicly report and externally verify how model architectures could enable opaque latent reasoning, and run chain-of-thought monitorability stress tests that third parties review.
Argues for model forensics, a neutral investigation discipline to determine whether a caught bad action reflects misalignment or a mistake, and reviews benign explanations for past concerning behaviors.
Argues that misalignment spreading during deployment is the most plausible near-term route to consistent adversarial misalignment, and critiques how current company risk reports, notably Anthropic's Mythos report, handle it.
Google DeepMind and UK AISI present a safety case sketch for control monitoring of frontier agent deployments, comparing asynchronous, semi-synchronous and fully-synchronous monitors on oversight, latency and harm-prevention conditions.
Argues that Thinking Machines' Tinker API, which keeps model weights server-side, could let AI labs limit researcher access to weights, reducing exfiltration and insider threats from humans or AI agents.
Proposes filtering narrow scheming data (control subversion strategies, eval results, misaligned model transcripts) from LLM training corpora to reduce misalignment risk, with a classifier and cost estimates.}
Argues LLM action monitors for misaligned agents can live in four places (agent scaffolds, cyber detection systems, LLM APIs, code review), comparing strengths, weaknesses and organisational ownership to prioritise deployment.
A UK AISI-backed research agenda outlining AI control priorities such as high-stakes control settings, research sabotage, covert malicious fine-tuning, and chain-of-thought monitoring, with a funding call.
A multi-organisation position paper arguing that monitoring chains of thought is a valuable but fragile AI safety opportunity and should be preserved and studied as a safety agenda.
Blog post surveying a defense-in-depth toolkit for mitigating AI scheming, covering architecture choices, control, white-box detection, prompting, and enforcement, and estimating risk reduction from roughly 25% to 10%.
Analyzes failure modes of instruction-following as an AGI alignment target, including shutdown resistance, principal authentication and jailbreaking, human-controlled proliferation, and conflicts from mixed training targets.
Argues AI safety research must plan for deploying known scheming models, analysing how catching escape attempts lets developers reduce risk via classifiers, policy training, and monitoring, and recommends a responsible disclosure and third-party deployment policy.
Prioritizes threats for AI control, arguing that undetected rogue internal deployments with escalated permissions are worse than self-exfiltration and should be the top priority to prevent or catch.
Argues that AI control may raise existential risk by shifting outcomes toward ineffective warning shots and takeover, since controlled incidents are unlikely to prompt serious action on alignment.
Sketches a structured safety case arguing an internally deployed LLM agent won't exfiltrate data, based on a red-team control evaluation of monitoring protocols and extrapolation to deployment.
Argues that a small internal safety team in a rushed, unreasonable AI lab should prioritise cheap, low-overhead control measures and external research support, and that such pessimistic scenarios deserve more planning focus.
Argues frontier AI companies are unlikely to produce high-assurance safety cases for existentially dangerous AI within short timelines, due to security, scheming-detection, and acceleration difficulties, and unlikely to pause unilaterally.
Apollo Research staff argue AI labs should publish detailed short-timeline (2027) plans, proposing minimal requirements: secure model weights and monitoring of scheming AIs, with faithful chain-of-thought and monitoring as core defenses.