Research on reliable, interpretable and steerable AI. anthropic.com
Anthropic tests automated adversarial stress-testing of safety monitors by having a Claude model write subtly misaligned training data that evades a training monitor, finding harmless-looking data that still teaches misalignment.
Anthropic has Claude autonomously train models to fix 10 categories of alignment failure, measured on public benchmarks, finding improvements without capability loss.
Tests whether automated AI alignment researchers can post-train models to reduce ten benchmarked failures like deception and sycophancy, outperforming human researchers and generalizing to held-out audits.
Nine autonomous Claude instances work five days on weak-to-strong supervision, testing whether LLMs can automate alignment research to scale oversight of smarter-than-human models.
Introduces an unsupervised method that elicits latent capabilities from pretrained models without external labels, matching or beating human-label training on TruthfulQA, GSM8k-verification, and reward modeling, motivated by superhuman alignment.
Introduces Constitutional AI, training a harmless assistant via self-critique, revision, and RL from AI feedback guided by a list of principles, with far fewer human labels.