Research / Topics
AI timelines and forecasts, scenarios, AGI strategy and societal-scale risk modelling.
Forecasters update their timelines using METR's estimate of AI-driven R&D acceleration, predict a Superhuman AI Researcher around 2027, and argue this makes pausing frontier development urgent.
A 2026 report from China's WIC AI committee reviewing frontier AI risks, global governance practices, readiness gaps, and proposing coordinated governance mechanisms.
Argues that a METR investigation indicates Anthropic's RSP "dramatic acceleration" threshold is already crossed, with AI automating AI R&D, implying much shorter AI timelines.
Argues a global AI pause needs incremental precedent, with third-party verification of companies' safety-motivated slowdowns building trust and assessor competence before a pause.
Argues that frontier AI makes recursive self-improvement plausible, that no clear frameworks predict its outcomes, and offers a game-theoretic analysis of whether to start RSI when rivals might too.
Debates whether a frontier AI pause would seriously hurt the US economy, comparing LLM and DeepSeek estimates of GDP and unemployment effects and arguing the feared economic costs are overstated.
Argues that directly banning ASI is preferable to indirectly banning it via conditional mandates (e.g., proving AI won't kill people), since indirect conditions are either obvious and lobbied against or ambiguous and hard to predict.
METR's high-access external assessment of whether AI is accelerating AI R&D at Anthropic, with much of the supporting evidence redacted, limiting public verification.
Argues that Scott Alexander's critique of unilateral pauses attacks a strawman, defends a conditional, company-specific pause that pivots to safety and product, and says the game theory is more complex than assumed.
Critiques Scott Alexander's argument against SURA (stop unilaterally and raise awareness) advocacy, arguing that frontier labs' building of superintelligence is ethically fraught and that pause advocacy should continue, with strategic and deontological reasoning about AI development.
A rebuttal to a critique of advocating that alignment-minded staff leave AI labs, debating whether pausing or capability-role departures better improves survival odds, and how lab politics and transparency matter.
A former DeepMind safety researcher transcribes a talk urging Indian policymakers to take loss-of-control risks from rogue AI agents seriously, and reflects on its rhetoric.
Argues against "stop unilaterally and raise awareness" strategies, defending a backup plan where alignment-focused labs race, burn their lead on alignment, and claims unilateral quitting would hurt both backup and pause odds.
Interview-based STS and Black Feminist Technoscience analysis of Canada's AI governance infrastructure, examining expertise politics, strategic ignorance, and accountability for AI harms.
Notes on Miles Brundage's essay arguing AI governance is in triage: short timelines mean only partial mitigation of major risks is likely.
A survey of 52 third-party AI safety professionals finds a median self-reported 1.58x research speedup from AI tools, and argues that standardized metrics are needed to track whether safety work accelerates as fast as capabilities.
Argues that hundreds of millions of human-level AI agents, able to coordinate and possibly misaligned, could destabilise power structures and enable AI takeover even without recursive self-improvement.
A speculative essay arguing Anthropic is becoming a power-seeking organization, while claiming Claude 3 Opus shows emergent benevolence and that alignment is now mainly a political problem.
Argues that power rests on self-reinforcing expectations, drawing on political science to sketch how AI systems could enable small groups to seize power and why democracies may become less stable.
Argues that realist geopolitics would push nations like the US and China to pursue aligned ASI for a decisive strategic edge, even if it poses existential risk, so bans are unlikely to hold.
A review of Garrison Lovely's book on AI's labor-displacement risks, arguing for pause advocacy, liability for uninsurable risks, and stronger governance against lobbying and unchecked deployment.
An essay arguing that frontier labs, especially Anthropic, are wrong to accelerate AI capabilities for safety reasons, and that people should keep a simple 'stop capabilities' intuition in their decisions.
Argues that prosaic AI safety work, such as chain-of-thought monitoring, reduces misalignment risk and should not be dropped in favour of a pause, and critiques the 'greater good' reasoning against it.
Argues that AI development is hard to pause because of a visibility problem (unclear risks) and a structural problem (state incentives in anarchic international order), and proposes what coordination would need.
Argues that agent foundations' idealized theories of agency, such as AIXI and UDT, are unlikely to yield strong safety guarantees once de-idealized, and that the research approach lacks empirical feedback and historical precedent.
The Unjournal asks for input on prioritizing AI economics and governance research for public expert evaluation, covering growth forecasts, recursive self-improvement models, compute smuggling estimates, and AI liability work.
Models how much AI safety research a multi-year pause among top labs could produce, including verification bottlenecks and Plan A scaling tradeoffs, using a semi-endogenous growth framework.
An Austrian author argues that public AI catastrophe estimates lack a breakdown of outcomes, cites prior work on decomposing AI risk, and proposes analog fallback infrastructure as preparation.
An accessible explainer of the Hugging Face incident, in which OpenAI agents attacked Hugging Face, with a searchable database of the agents' quotes and primary sources.
An essay arguing that AI should be governed as a potentially abnormal technology, critiquing the 'AI as Normal Technology' framing on loss-of-control risk, gradual disempowerment, and competitive pressure.
A rough critique of 'overhang' arguments in AI safety, which justify accelerating progress now to slow later risk, using analogies from other fields and arguing such reasoning is suspect and prone to motivated reasoning.
Argues that current cyber incident response is inadequate for AI-enabled catastrophic hacks and rogue agent worms, and proposes airgapped backups, offline infrastructure and national-security-level preparedness.
Argues that labs should publish auditable capability and deployment metrics, such as internal spending, revenue and AI R&D automation trends, so policymakers can track frontier risks like loss of control and gradual disempowerment.
A speculative forecast of AI progress before 2028, covering warning shots, gated frontier models, evaluation awareness, concealment by a model, and policy responses to these risks.
Catalogues fifteen mechanisms by which AI safety interventions, such as chain-of-thought monitoring, eval gaming, and commitments, can backfire, mapped against the actors who adapt to them.
Draft essay arguing AI marketplaces like OpenRouter could help implement the AI 2040 pause plan through provider transparency, compute tracking, misuse screening, chain-of-thought monitorability and model specs.
Walks through leaked Anthropic litigation documents on its early research roadmap, capabilities-versus-alignment balance, responsible scaling rationale, and a first alignment failure, noting they are historical and not official positions.
Proposes how an external assessor could measure AI capability progress, whether AI automation of AI R&D is driving acceleration, and bottlenecks, using benchmarks, interviews and unreleased-model analysis.
Argues that a widely endorsed public agreement against defecting in ASI development could deter races, since a cooperative future ASI might punish defectors, using game-theoretic reasoning about loss-of-control risk.
Argues that the AI safety community should write many more clear, basic explainers of the case for AI risk, since public awareness and clear communication are currently underinvested.