An operationalization of opaque serial depth
Operationalizes NLS depth, a measure of unverbalized serial computation in models, proposing criteria for natural-language-rooted nodes to track CoT monitorability risks from architectural changes.
Research
Alignment Forum
Discussion and research on AI alignment. alignmentforum.org
AI (574)Interpretability (ML & AI) (73)AI Control (51)AI Evaluations (31)Chain-of-Thought Alignment (27)AI-Assisted Alignment (16)AI Risk (13)AI Takeoff (13)AI Governance (12)AI Timelines (10)Sandbagging (AI) (10)Debate (AI safety technique) (7)AI Auditing (6)Alignment Pretraining (6)Dealmaking (AI) (6)AI Psychology (5)AI Alignment Fieldbuilding (4)AI Rights / Welfare (4)AI Risk Concrete Stories (4)AI Safety Cases (4)Aligned AI Proposals (4)Jailbreaking (AIs) (3)Open Source AI (3)Regulation and AI Risk (3)Threat Models (AI) (3)AI Alignment Intro Materials (2)AI Safety (2)AI Safety Camp (2)AI Safety Public Materials (2)AIXI (2)Grants & Fundraising Opportunities (2)Guaranteed Safe AI (2)OpenAI (2)Slowing Down AI (2)AI Benchmarking (1)AI Capabilities (1)AI Development Pause (1)AI Lab Self-Immolation (1)AI Success Models (1)Aligned AI Role-Model Fiction (1)Frontier AI Companies (1)Human-AI Safety (1)Neuromorphic AI (1)Power Seeking (AI) (1)Prosaic Alignment (1)Truthful AI (1)
Focus All
Topic Evals & standards + Control
Source AF · 10
Date Any time
2 items
Proposal for tracking the effects of architecture on monitorability
Proposes that AI companies publicly report and externally verify how model architectures could enable opaque latent reasoning, and run chain-of-thought monitorability stress tests that third parties review.
1 item
The Case for Model Forensics
Argues for model forensics, a neutral investigation discipline to determine whether a caught bad action reflects misalignment or a mistake, and reviews benign explanations for past concerning behaviors.
1 item
Risk reports need to address deployment-time spread of misalignment
Argues that misalignment spreading during deployment is the most plausible near-term route to consistent adversarial misalignment, and critiques how current company risk reports, notably Anthropic's Mythos report, handle it.
1 item
Can governments quickly and cheaply slow AI training?
Analyzes whether inference-verification measures like bandwidth limits, memory wipes and proof of work could let governments quickly slow AI training, concluding current prototypes are likely ineffective but some measures might buy time.
1 item
Practical challenges of control monitoring in frontier AI deployments
Google DeepMind and UK AISI present a safety case sketch for control monitoring of frontier agent deployments, comparing asynchronous, semi-synchronous and fully-synchronous monitors on oversight, latency and harm-prevention conditions.
1 item
Research Areas in AI Control (The Alignment Project by UK AISI)
A UK AISI-backed research agenda outlining AI control priorities such as high-stakes control settings, research sabotage, covert malicious fine-tuning, and chain-of-thought monitoring, with a funding call.
1 item
Handling schemers if shutdown is not an option
Argues AI safety research must plan for deploying known scheming models, analysing how catching escape attempts lets developers reduce risk via classifiers, policy training, and monitoring, and recommends a responsible disclosure and third-party deployment policy.
1 item
A sketch of an AI control safety case
Sketches a structured safety case arguing an internally deployed LLM agent won't exfiltrate data, based on a red-team control evaluation of monitoring protocols and extrapolation to deployment.
1 item
AI companies are unlikely to make high-assurance safety cases if timelines are short
Argues frontier AI companies are unlikely to produce high-assurance safety cases for existentially dangerous AI within short timelines, due to security, scheming-detection, and acceleration difficulties, and unlikely to pause unilaterally.