Ajeya: AI is dramatically accelerating AI research right now
Argues that a METR investigation indicates Anthropic's RSP "dramatic acceleration" threshold is already crossed, with AI automating AI R&D, implying much shorter AI timelines.
Argues that a METR investigation indicates Anthropic's RSP "dramatic acceleration" threshold is already crossed, with AI automating AI R&D, implying much shorter AI timelines.
Reviews open problems in evaluating general-purpose chatbots for mental-health risks like sycophancy and delusion reinforcement, and argues for real-world data access and evidence-based policy.
Introduces BMMS, a policy-conditioned suite that scores LLM compliance with published rulebooks, with an attribution-based admission test; finds GPT-4o fails 70.7% of its Model Spec hard constraints versus 0% and 9.1% for Claude models.
METR's high-access external assessment of whether AI is accelerating AI R&D at Anthropic, with much of the supporting evidence redacted, limiting public verification.
METR reviews Anthropic's risk report on automated AI R&D, finding catastrophic risk was low but the early-warning threshold may have been crossed, and that the report understates acceleration.