Trustworthy agents in practice
Anthropic explains how Claude agents work, applies five trustworthy-agent principles to product decisions, and describes defenses across model, harness, tools and environment, plus what industry and governments can do.
Research / Sources
Anthropic
Research on reliable, interpretable and steerable AI. anthropic.com
Focus All
Topic Lab practice
Date Any time
Anthropic's Pilot Sabotage Risk Report
Anthropic's pilot risk report assesses misalignment risk from its deployed models as of summer 2025, concluding risk of catastrophic sabotage is very low but not negligible, under an affirmative-case framework.
Why do we take LLMs seriously as a potential source of biorisk?
Anthropic explains why it treats LLMs as a potential biorisk source, motivating dangerous-capability evaluations and CBRN safeguards under its Responsible Scaling Policy after activating ASL-3 protections for Claude Opus 4.
Developing nuclear safeguards for AI
Anthropic and the NNSA co-developed a classifier that separates concerning from benign nuclear-related conversations at 96% preliminary accuracy, now deployed on Claude traffic to monitor misuse.
Introducing the Anthropic Fellows Program for AI Safety Research
Announces Anthropic's pilot fellowship funding and mentoring 10-15 researchers for six months on AI safety topics like adversarial robustness, dangerous capability evals, and scalable oversight.
Three Sketches of ASL-4 Safety Case Components
Anthropic sketches hypothetical safety-case arguments for ASL-4 models with strategic and agentic capabilities, covering misuse, autonomous replication and autonomous AI research risks under its RSP.
Collective Constitutional AI : Aligning a language model with public input
Anthropic and the Collective Intelligence Project had about 1,000 Americans draft an AI constitution, then trained a model with Constitutional AI against it and compared it to the in-house version.
Predictability and Surprise in Large Generative Models
Argues that large generative models combine predictable aggregate loss with unpredictable specific capabilities and harms, analyses the policy implications, runs two experiments on unpredictability-driven harms, and proposes interventions.