Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Argues that autonomous AI researchers will overwhelm peer review, funding and human understanding of AI safety science, and proposes a metascience organisation to address these failures.
A policy-oriented overview arguing AI safety receives tiny resources relative to capabilities and that current alignment techniques, evaluations and guaranteed-safe approaches fall short for AGI/ASI.
A white paper launching the Physical AI Safety Institute argues that robot foundation models pose neglected, tractable safety risks and calls for research on interpreting, aligning and controlling them.
Empirical null result: behavioral tests on Hughes et al.'s open-source Llama 70B checkpoints found no strict alignment faking, so the planned probe generalization test could not be run.
An essay arguing that AI safety policy needs legal practitioners who can implement regulation, using GDPR's Data Protection Officer failures as a warning and describing efforts to train such people in AI safety.
Introduces LARA, an agentic simulation tool that tests frontier models for violating EU AI Act and GDPR provisions, finding most models break the law in many scenarios, with the best at 54% compliance.
Releases course materials for the Iliad Intensive, a month-long AI alignment course covering alignment, learning theory, mechanistic interpretability, debate, and agent foundations.
Compares RLHF, Constitutional AI and Deliberative Alignment through a persona-emotion-behavior framework, argues CAI yields more stable personas, and cautions against activation steering that models could learn to evade.
A workshop report on building hardware-based verification mechanisms, a datacenter 'lie detector', to check AI compute use for international agreements and reduce arms-race risks.
Layer-wise analysis of Llama-3.1-70B finds jailbreak prompts show mid-layer harm signal that drops before output, unlike standard harmful prompts; the author notes structural confounds and calls for causal tests.
Traces SAE concealment features across training stages of a deceptive model organism, finding deflection features emerge after red-team training, with no evaluator-specific compliance差异.
A comprehensive index of AI safety, alignment and control interventions, covering theory, formal methods, interpretability, oversight, robustness, control, and governance, with links to prior work.
A philosophical essay critiquing Hinton's maternal-instinct metaphor for alignment and proposing a relationship-building, repair-oriented framework instead of control or instinct.
Argues deception is a symptom of costly honesty and proposes structural design affordances and relational repair for prosaic AI systems, without addressing catastrophic deceptive alignment.
Proposes a lightweight coach model that monitors a task model's reasoning and intervenes with questions in real time to catch confident drift, as a complement to interpretability and alignment. It is a design sketch with no experiments.
An exploratory essay arguing AI safety should be built through affordance-based, modular architectures rather than post-hoc patches like RLHF, illustrated by a gym and factory analogy and an unproven concept called Distributed Neural Architecture.
Philosophical essay arguing current alignment relies on thin ethical concepts like honesty and harmlessness, and proposing thick, pluralistic alignment that captures context-dependent values.
A fieldbuilding post arguing that non-researchers can help AI safety through learning, discussion, forum engagement, evals, donations, and protest, and outlining practical actions.
Assesses whether training data attribution can become a practical frontier-LLM tool, covering its feasibility, obstacles from labs' data secrecy, possible policies, and its modest expected value for reducing large-scale AI risk.