AI research and discussion on LessWrong. www.lesswrong.com
An essay arguing that AI-transition planning should preserve liberal-republican values such as non-domination and popular government, not just survival or aligned benevolent rule, and criticising both the democracy-focused and takeover-accepting camps.
A dialogue defending Substrate Needs Convergence, the claim that AGI alignment is impossible because evolutionary pressures override engineered goals, and debating its political and philosophical implications for AI x-risk policy.
Argues that 'alignment' conflates two bars, avoiding rogue behaviour versus fully internalising human values, and frames the goal of AI transition as a high-stakes succession problem with fragile value.
A personal backchaining essay on what positive AGI futures require, arguing for pluralistic, individually controlled and aligned AI plus a hardened, defensive-acceleration society against catastrophic risks.
A personal catalogue of counterarguments to corrigibility as an AI alignment target, covering formalism, anti-naturalness, power concentration, principal selection, and alternatives like honest AI.
A translated essay argues that value alignment is a pseudo-concept, citing undefined values, unrepresented humanity, two-way influence, and corporate 'alignment theatre', and proposes deliberative governance instead.
A debate between Seth Herd and cousin_it on whether concentrating control of a powerful instruction-following AI in a few people is riskier than distributing it, with arguments from history and psychology.
Argues that concentrating control of aligned ASI in one or a few humans could be safer than distributed AI, because secure absolute power may make people more prosocial over time, and assesses the risks.
Develops a conceptual framework for gradual human disempowerment by AI-driven societal systems, analysing processes like competition, standardization and centralization, and the mechanisms that could counter them.
Argues that making AI corrigible is undesirable because it would concentrate control in whichever humans command the AI, and that corrigibility training may backfire by teaching models they are untrustworthy.
A LessWrong essay rebuts Narayanan and Kapoor's 'AI as normal technology' thesis, arguing AI is categorically transformative and that diffusion-speed and intelligence-measurement arguments understate its risks and impact.
Applies Leveson's systems-safety thinking to AI loss of control, modelling how breaches at each oversight level can escalate into vicious cycles and how AI could also strengthen governance.
A grantmaker proposes a cost-effectiveness unit measured in future-improvement per dollar, illustrating BOTEC comparisons of AI safety donation opportunities such as super PACs, nonprofits and Alex Bores' campaign.
A reader's guide to Eric Drexler's writing on AI strategy, arguing against anthropomorphising AI, favouring system-level design thinking, and encouraging readers to reinterpret his ideas and examine open questions.
A conceptual essay arguing that single-objective alignment is the wrong target and proposing responsibility-based, friction-preserving, controlled delegation of control to AI systems as an alternative.
Conceptual notes framing AI-driven power concentration as a 2x2 of AI versus human power and emergent versus power-seeking dynamics, and as erosion of legacy checks and balances.
Argues that the traditional Yudkowsky-style pivotal act requires suppressing humanity and proposes a non-coercive alternative: an aligned ASI demonstrates its power, then hands control back and promotes voluntary AI regulation.
Argues that AGI could enable lasting path-dependent lock-in of values and power through AGI-enforced institutions, immortality, value design, and self-modification, making steering work valuable.
Speculative strategic essay arguing that permanent human disempowerment without extinction is a likely baseline outcome of AGI, and discussing AI-driven pauses on superintelligence and value alignment.
Proposes eight conceptual pathways by which gradual disempowerment could become permanent even if AIs are shutdownable, covering principal choices, legal lock-in, value drift and coordination failure.
An essay arguing that solving technical alignment via faithful, obedient AI would still leave transition risks like arms races, surveillance, misuse, internal deployment, and safety underinvestment, and proposing mitigations.
A transcript of a talk on AI welfare and moral status, covering consciousness, stakes, over-attribution, and how to act under uncertainty; it is philosophical rather than a safety technique.}
Argues that anthropomorphic AI intuitions may help public alarm about AGI risk, while stressing that human-like agents will lack prosocial instincts and so remain dangerous.
A plain-language summary of the gradual disempowerment paper, which argues that AI could erode human influence over economy and society through incremental, misaligned incentives rather than a sudden takeover.
Summarises a paper arguing that gradual, competitive replacement of humans by AI could erode human control over society even if alignment succeeds, and that no good mitigations are yet known.
Argues that a US race to artificial superintelligence is self-defeating due to great-power conflict, loss-of-control, and power-concentration risks, and proposes international verification-based restraint as strategically sound.
An opinion essay arguing that AI-driven societal transition is dangerous without a plan, and that defining and aggregating human values is conceptually very hard, so deploying ASI first is reckless.
An essay arguing that AI alignment alone cannot secure humanity's goals, so the priority should be human alignment, meaning coordination or coercion to stop reckless superintelligence development.