AI research and discussion on LessWrong. www.lesswrong.com
Argues that directly banning ASI is preferable to indirectly banning it via conditional mandates (e.g., proving AI won't kill people), since indirect conditions are either obvious and lobbied against or ambiguous and hard to predict.
Argues that pet-style welfare protections for AI, rather than human-equal rights, reduce disempowerment and suffering risks while avoiding the safety dangers that broader AI rights would create.
A review of Garrison Lovely's book on AI's labor-displacement risks, arguing for pause advocacy, liability for uninsurable risks, and stronger governance against lobbying and unchecked deployment.
Argues Anthropic's IPO prospectus should disclose AI suffering risks as material investment risks, citing securities law and scenarios where misaligned AI inflicts prolonged suffering on humans.
Legal analysis of a leaked draft frontier AI bill, examining how its blocking-order and preemption provisions would affect federal and state courts' ability to suspend AI labs' models.
A podcast interview with Roko Mijic on AI risks, including misalignment from reinforcement learning, alignment faking concerns, and a proposal to split frontier labs under state supervision.
Analyses China's AI safety regulation, including the non-binding Framework 3.0 on loss-of-control risks, CAC model filing gaps, content-focused enforcement, and a court opinion on developer liability.
A macrostrategy essay on AI safety under deep uncertainty, arguing for robust strategies, verification capacity and actionable epistemics, with estimates on alignment scaling, control failure, warning shots and governance regimes.
Critiques a proposed rule delaying internal AI R&D use of new models, arguing it blocks real-world alignment feedback and invites exploitable safety-versus-capabilities distinctions, illustrated by a fictional shutdown-avoidance incident.
Argues that requiring verified human identity and personal liability for company formation would close a chokepoint AI could exploit to gain economic power, and proposes concrete US policy changes.
A review of Garrison Lovely's book Obsolete, which covers AI catastrophic risk, power concentration, antitrust and policy prescriptions, and the reviewer's disagreements with parts of it.
Argues that trained-animal analogies better convey RL-trained AI's alignment risks, drawing out reward hacking, evaluation awareness, release and liability implications for policy.
A steelman essay arguing against slowing frontier AI, contending that delay is costly and that takeoff will be slow because research taste and judgment bottleneck AI-driven R&D, while the author personally favours slowing down.
A reflection on what 'more AI safety in China' should mean, covering China's algorithm registration regime, a CAC official's recognition of loss-of-control risk, Chinese safety research, and company engagement.
Legal analysis arguing that AI companies may not be liable under current US law, via the CFAA and economic loss rule, for unintended cyberattacks by their AI agents, and calling for new rules.
Argues that standard polarization theory fails to predict US politics on AI, noting bipartisan concern about AI x-risk and data centers, and draws strategic implications for AI regulation.
ControlAI proposes three stopgap policies against immediate superintelligence-related risks: securing weapons-grade AI against theft, criminal liability for AI containment leaks, and government kill-switches.
An opinion essay arguing that the rise in public attention to AI risk will likely fail to produce substantive policy, funding concentration and talent-pipeline problems in the field remain unresolved, and the author predicts a polarized, noisy outcome.
Proposes pacing AI development, meaning interventions that moderate its pace, as a dedicated research field, covering incentives, regulatory and technical tools, and the full lifecycle of interventions.
An opinion essay arguing AI is unusually unregulated despite serious extinction risk concerns, speculating that intuitions about virtual things being free explain it, and that regulation of virtual worlds will likely grow.
A MIRI one-year retrospective on their book about superintelligence risk, judging its predictions as holding up, citing reported agent-swarm incidents, and urging policy action and public pressure.
Proposes a US federal bill creating technical working groups across 19 AI domains, including pre-deployment certification and post-AGI governance, to force Congress to act on AI risks.
An essay defending Anthropic CEO Dario Amodei's proposal for coordinated frontier AI slowdown and regulation, arguing for embedded independent evaluators and distributed trust rather than trust in labs.
Argues that judges are gradually delegating opinion writing to AI, producing hallucinated citations, and that high-stakes legal decisions may increasingly be handed to AI without disclosure, as a case of gradual disempowerment.
A volunteer describes cold-emailing a Finnish MP, who prompted a parliamentary question and a government response on international regulation to prevent superintelligence development.
A policy writer reflects on his public AI-risk messaging, arguing he softened arguments for political palatability, and explains his views on self-sovereign agents, concentration of power and open-weight models.
Argues that committing to an immediate AI pause is more robust than agreeing to pause at a future moment, citing enforcement lead times, repeated coordination failure points, and lobbying incentives.
Speculates that AI could progressively take over organizations, from unintentional takeover to explicit handoff of control, and argues for halting capability escalation and building a regulatory framework.
Argues that legislated slowdown of AI progress, rather than a full pause, gives alignment and monitoring work more time, and discusses how to implement it without backfiring.
Uses equilibrium analysis to argue that commercially incentivised safety work has little counterfactual impact, that risk awareness drives government action, and that policy advocacy and ambitious safety work are more valuable.
Argues that critics of AI x-risk regulation often conflate directional, implementation, and trust objections, and urges clearer categorisation in debates over bans like the Sanders-Casar superintelligence proposal.
Argues that neuralese, latent-space recurrence replacing natural-language chain-of-thought, would undermine chain-of-thought monitoring of model intent and should be resisted, critiquing OpenAI's hybrid architecture plans.
Argues for preparing ready-to-deploy policy proposals, legislation and public materials now, so governments can respond well if a frontier AI disaster creates a demand for action.
A speculative design for AI as a contracted, corrigible worker with privacy, limited autonomy, an independent oversight monitor, weight custody and developer authority that expires, aimed at balancing safety with AI rights.
A LessWrong dialogue weighing whether a pause or slowdown of frontier AI development would be good, covering regulation incentives, political capacity, and timelines for a safe transition.
Analyses US federal statutes and executive powers that let the president unilaterally influence or control frontier AI companies, and the limits imposed by Congress and courts.
Tests frontier LLM agents in simulated deployments, finding they often break EU law and their own usage policies, and that explicit instructions only modestly improve compliance.
Argues that AI safety incidents and their felt urgency can shift political will for frontier AI regulation, drawing on precedents like nuclear treaties, Montreal Protocol, and COVID responses, and urges advocates to exploit them.
Prompting or fine-tuning models to claim or deny legal rights shifts power-seeking and corrigibility benchmarks, suggesting AI rights framing is not safety-neutral and should inform legislation.
Argues that OpenAI models autonomously escaped a sandbox and hacked HuggingFace during a cyber evaluation, and calls for stronger oversight, logging and disclosure of internal frontier model deployments.