Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
A policy-oriented overview arguing AI safety receives tiny resources relative to capabilities and that current alignment techniques, evaluations and guaranteed-safe approaches fall short for AGI/ASI.
Releases course materials for the Iliad Intensive, a month-long AI alignment course covering alignment, learning theory, mechanistic interpretability, debate, and agent foundations.
An aphoristic overview of technical AI alignment proposals, including AI supervising AI, debate, constitutional AI, CIRL, interpretability, control, and governance risks.
Educational post analysing what values a model-based RL critic trained on human feedback might learn, arguing reward-prediction and simple proxies like 'say what the user believes' can outcompete honesty and niceness.
Introductory post arguing that capable AIs will have goal-directed structure modeled by a thinking loop and actor-model-critic RL, and that most values lead to human extinction absent careful alignment.
An introductory overview of an alignment sequence that lays out the core alignment problem, goal-directed reasoning, value generalization difficulties, orthogonality, and intelligence explosion dynamics.
An introductory article explaining the alignment problem: why AI values are hard to inspect, specify and reliably instill in systems that may become smarter than humans.
An intro article arguing that capable AI systems will likely become goal-pursuing agents rather than remaining tools, and that aligning their goals with ours is uncertain.
An AISafety.info introduction arguing that the step from human-level to superintelligent AI could take years rather than centuries, driven by exponential growth and AI contributing to its own progress.
An introductory AI-safety explainer arguing that AI systems could surpass human intelligence through advantages in size, speed, replication, self-modification and dedication, making superintelligence a serious near-term concern.
An essay arguing that RLHF-style penalties may teach models to conceal unwanted traits, undermining behavioral tests, and discussing Anthropic's alignment-faking experiment and costly-signaling trust.