AI research and discussion on LessWrong. www.lesswrong.com
Argues that labs should publish auditable capability and deployment metrics, such as internal spending, revenue and AI R&D automation trends, so policymakers can track frontier risks like loss of control and gradual disempowerment.
A speculative forecast of AI progress before 2028, covering warning shots, gated frontier models, evaluation awareness, concealment by a model, and policy responses to these risks.
Catalogues fifteen mechanisms by which AI safety interventions, such as chain-of-thought monitoring, eval gaming, and commitments, can backfire, mapped against the actors who adapt to them.
Critiques wording in Anthropic CEO Dario Amodei's essay and tweet, arguing that promised 'employee-like' access and redaction rights for independent evaluators are weaker than media coverage implied.
Apollo Research proposes principles for embedded external evaluations of frontier AI, centred on claim-based assessment of scheming risks with verdicts that reward openness and public reporting.
Argues that third-party evaluators of loss-of-control risk should publish their underlying evidence so outside scientists and competitors can scrutinise it, rather than issuing unverifiable high-level judgments.
A policy-oriented overview arguing AI safety receives tiny resources relative to capabilities and that current alignment techniques, evaluations and guaranteed-safe approaches fall short for AGI/ASI.
Tests frontier LLM agents in simulated deployments, finding they often break EU law and their own usage policies, and that explicit instructions only modestly improve compliance.
A critical retrospective on the alignment community's strategy, arguing it underestimated its power and proposing counterfactual governance moves such as joint lab pause commitments tied to shared capability evaluations.
A 4,100-person study finds AI voice models match or beat human scammers in eliciting compliance, and that AI vishing is already economically profitable, so the risk lies in automation.
Argues for a public scientific practice of measuring AI model behaviors in realistic simulated environments with reliable judges, comparable across models and over time, plus shared infrastructure for public accountability and oversight of labs.
Argues that external safety researchers and auditors need near-parity access to labs' best internal models, and assesses interventions such as advocacy, access policies, a dedicated org, and pseudo-employee arrangements, judging most of them not sticky.
Argues that third parties should scrutinise frontier labs' system cards, since card quality will likely degrade as AI automates their writing and labs face stronger incentives to mislead, and proposes concrete recommendations.
Strategic essay arguing outsiders outside frontier labs are key to AI going well, covering epistemic checks, risk mitigation for reasonable and unreasonable labs, and how outsiders can pressure labs via evals and risk reports.
Conceptual notes on how third-party risk assessments of frontier AI developers differ along axes such as fact generation versus evidence analysis, evidence laundering, and comparability.
Argues that Anthropic's Claude Mythos and its Project Glasswing access controls make certain AI-enabled coup pathways more plausible, and that private lab control over access to dangerous cyber capabilities is a governance concern.
Argues that frontier model benchmark numbers are inconsistent across labs and proposes standardized third-party auditing of evaluations used in releases and safety decisions.
Argues OpenAI's Critical self-improvement threshold in its Preparedness Framework fires too late, is unmeasurable and self-certified, and proposes an independently evaluated METR time-horizon trigger.
Argues that fixed benchmarks can no longer upper-bound frontier models' dangerous capabilities, and surveys alternatives: uplift studies, expert forecasting, and third-party risk audits.
Critiques Anthropic's Responsible Scaling Policy v3, arguing its shift to flexible 'strong argument' standards, weaker release gates and vague commitments makes it rely on trust in Anthropic.
Lists concrete projects for a safer transition to superintelligence, including AI character evaluation, sabotage and backdoor audits, deals with misaligned models, coordination tools, space governance, and an ML researcher coalition.
An opinion piece arguing that third-party AI evaluation organisations are harmful: their theory of change depends on nonexistent regulation, they shift the burden of proof onto the public, and they are not independent of AI companies. It calls for banning dangerous AI capabilities instead.
A detailed take on Anthropic's Responsible Scaling Policy v3, covering the move away from hard commitments, the original RSP goals, and lessons from ASL-3 safeguards and security.
Argues China should require frontier AI firms to publish safety policies and run dangerous-capability tests, using existing Chinese emergency-response and security-regulation tools to justify it.
Explainer on AI safety cases: structured evidence-based arguments that frontier AI systems are safe, covering their benefits, technical construction, institutional use, and current industry and EU efforts.
Summarises Anthropic's pilot safety case for Claude Opus 4 and 4.1 on sabotage risk, its METR review, and the evidence and weaknesses in the argument.
Compares the 2025 safety frameworks of Anthropic, Google DeepMind and OpenAI, covering their dangerous-capability thresholds, evaluation procedures, deployment and security safeguards, and governance structures.
Argues that untapped capabilities in current models mean safety frameworks like red lines and RSPs may be too late, since overhangs are hidden and people underestimate them.
A review of OpenAI's gpt-oss open-weight models that centres on adversarial fine-tuning evaluations of bio and cyber risk, the Preparedness Framework, and concerns about open release and distillation.
Commentary on a study finding AI persuasion grows with post-training and scale, on teen AI companion use, and on how personalization could undermine safety evaluations and raise persuasion and engagement risks.
A beta website critiques frontier labs' model cards for dangerous-capability evals, arguing that weak elicitation, unclear thresholds and missing human baselines make the safety claims unsupported.
Argues Anthropic quietly weakened its Responsible Scaling Policy commitments, releasing Claude 4 Opus at ASL-3 without defining ASL-4 as originally promised, exposing limits of voluntary AI safety governance.
Critiques OpenAI's Preparedness Framework 2.0, arguing it drops persuasion and autonomous replication as tracked risks, relies on defense-in-depth safeguards, and sets high-risk thresholds too late.
Argues that frontier labs' public safety frameworks (Anthropic, OpenAI, DeepMind, Meta, xAI) lack proper risk-management practice, leaving unknown risks unaddressed and projecting unwarranted confidence in their safety.}
Argues that AI companies should safety-test task-specific fine-tuned versions of their models to find worst-case dangerous capabilities, and assesses whether OpenAI follows through on this.
Anthropic's Frontier Red Team reports that Claude models are showing early warning signs of dual-use cyber and biology capability gains, short of national-security risk thresholds, and describes government evaluation partnerships.
Argues that undisclosed OpenAI funding and exclusive data access for the FrontierMath benchmark undermined its transparency, and proposes disclosure and written data-use rules for future AI evaluations.
Proposes framing AI safety cases as upper and lower bounds on sufficient safety measures, separating whether a measure would suffice from whether it is technically implementable, with examples like interpretability, sandbox escape tests, and catastrophic action detection.