Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Estimates how far zero-knowledge proofs of LLM inference are from cost-competitive with auditing for verifying AI compute in international agreements, measuring roughly 1,700-3,800x proving overhead and hash-function compatibility issues.
Proposes a shared, public-cluster observatory of GPU telemetry and other physical signals to verify AI training runs, as a first measurement layer for frontier AI governance.
Analyses US federal statutes and executive powers that let the president unilaterally influence or control frontier AI companies, and the limits imposed by Congress and courts.
Proposes a network-based verification design, traffic shaping plus randomized routing, that estimates it would make covert frontier training runs economically infeasible and so support treaty compliance.
Proposes a taxonomy for verifying and enforcing international AI pauses via compute monitoring, estimating that pauses are enforceable for training runs above roughly 10k H100-equivalents absent concealment.
Proposes and tests a side-channel method that estimates LLM training FLOPs from Jetson GPU power and memory readings, to help verify compute thresholds in AI treaties, reaching about 4.5% mean error.
Proposes a low-trust reference architecture for verifying AI compute, using network taps, air-gapped or ZKP-based evaluation, and workload evidence capture to check treaty compliance.
Argues that compute verification is key to AI coordination, that die-level verification is too slow for short timelines, and that board- and firmware-level verification for Rubin and Feynman chips is more tractable and must start now.
Argues that FLOP/s and bandwidth proxies no longer track AI compute well and proposes architecture-specific compute scaling models, to improve forecasts and compute governance thresholds.
A workshop report on building hardware-based verification mechanisms, a datacenter 'lie detector', to check AI compute use for international agreements and reduce arms-race risks.
Argues that hardware-enabled mechanisms built into AI chips could verify and enforce AI regulations, surveying technical challenges, deployment timelines, and policy scenarios such as export-control location verification and US-China treaties.
Proposes how an international agreement to halt superintelligence development could be verified and enforced, through monitoring AI chip supply chains, data centers, and researchers, plus deterrence measures.
Explains the MIRI team's reasoning for FLOP thresholds in an illustrative international agreement to halt superintelligence development, arguing for conservative limits given uncertainty about dangerous capabilities.
MIRI's Technical Governance Team proposes an international agreement to halt frontier AI advancement toward superintelligence via compute limits, chip tracking, and research restrictions, justified by misalignment and catastrophic risk.
Essay arguing that falling AI development costs and easy model theft will lead to proliferation of offensive capabilities, drawing analogies to cryptography and estimating training cost trends for dangerous AI.
Compares feasibility and strategy of Global Shutdown versus Global Controlled Takeoff for AI, arguing controlled takeoff is not clearly more tractable and that removing economic pressure to rush is key.
Argues AI safety governance research is too academic and lacks advocacy, and catalogs eleven unfleshed policy ideas (windfall clauses, liability, compute monitoring, incident hotlines) with suggestions for drafting them.
MIRI's technical governance team presents a research agenda on AI strategy, mapping four geopolitical scenarios for advanced AI and favouring an internationally coordinated Off Switch and Halt to reduce extinction risk.
Proposes and reviews a conditional international AI safety treaty with compute thresholds, mandatory model audits, and AISI-led pause authority to address loss-of-control risks from advanced AI.
Zvi Shapira critiques the Superintelligence Strategy paper's MAIM deterrence proposal, examining its nonproliferation, competitiveness, and loss-of-control assumptions and whether they hold.
AE Studio argues alignment techniques can boost capabilities, so alignment is a competitive advantage, and urges US policymakers to fund neglected alignment research and adopt incentives for it.
Proposes a unified regulatory schedule for frontier model training, with pre-registered plans, safety evaluations before approval, and market-access incentives to coordinate safer ASI development.