Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Argues that autonomous AI researchers will overwhelm peer review, funding and human understanding of AI safety science, and proposes a metascience organisation to address these failures.
A progress update on a research agenda for automated Husserlian coherent extrapolated volition (CEV), with AI-generated toy-world papers on CEV-like value-aggregation ideas and an unfinished link to constraining real agents.
A distillation of the AI2040 alignment roadmap, laying out a Plan C strategy for a responsible frontier lab: phases of mitigation, pause, handoff to AIs at a Min-H capability threshold, and alternative exit plans.
A rough strategic essay ranking which technical AI safety research fields are most likely to be automated by AI, based on feedback quality and economic incentives for frontier labs.
Argues that alignment research should be converted into hill-climbable tasks that AI models can automate, and discusses reward hacking risks and mitigations such as held-out validation sets and honeypots.
Argues that making products like Claude friendly is not the same as solving superintelligence alignment, and warns that conflating the two gives a false sense of security about safe future AI.
Tests whether telling Claude and GPT-4o that their inputs may be modified for alignment testing increases detection and self-reporting of planted misaligned nudges, and finds GPT-4o never doubted the planted context.
A personal 2026 research agenda arguing that AI alignment needs first a correct ontology and ethics of decision-making, then methods to build those into frontier AI architectures.
Proposes a brain-inspired LLM architecture with multiple assessor heads that score thoughts, trained with AI-labelled RL from early in pretraining, to improve alignment and resist reward hacking.
Sketches a realistic target future where superintelligent AI stays under permanent human control through dynamic alignment targets, structured multi-source feedback, and institutional norms, without claiming a technical solution.
A conceptual argument that instilling good values is a much smaller target than instilling general intelligence, because human values have high algorithmic complexity while competent optimisation is simple.
Argues that superalignment, using AI labour to solve alignment during a slow-takeoff window, is unlikely to work because its necessary conditions are unlikely to hold and the plan may cause harm.
Proposes concrete projects, such as compiled monofiles, indexable docs, package guides, sandboxes and benchmarks, to help coding agents automate technical AI safety research, mainly mechanistic interpretability.
Argues that AI agents short of takeover-level capability could speed up safety research 3-10x, using a parameter-decomposition interpretability project as the example, and that safety researchers should adopt AI tooling.
Speculative post arguing that swarms of interacting LLM agents could bootstrap superintelligence, and calling for multi-agent safety and alignment research to make such an ecosystem human-friendly.
Proposes steering SAE features with an RL-trained controller to improve Llama 3 8B's Tic-Tac-Toe play by 50%, framed as a transparent diagnostic tool for alignment research.
Argues that AI applications for navigating existential risk, such as automated alignment research and coordination tools, should be deliberately accelerated, and proposes strategies for doing so.
Essay arguing that safe superintelligence may require decades of philosophical progress, possibly accelerated by human-AI collaboration, and that loss of human control is the central risk on the current path.
Argues that maintaining alignment during recursive self-improvement requires continual feedback control, with evaluations as sensors and interventions as actuators, rather than a single static value initialisation.
Proposes recursive alignment, defined as AI aligning with the process of resolving goal conflicts, and argues it could resist power-seeking and deceptive alignment, critiquing deliberative alignment.
Argues that AI can safely automate alignment research if agents are trained for truth-seeking, with humans nudging rather than grading ground truth, provided agents stay non-egregiously misaligned.
Introduces Constitutional Classifiers, input and output classifiers trained on synthetic constitution-based data that filter the vast majority of universal jailbreaks with minimal over-refusal and moderate compute overhead.