Research / Sources
Research on reliable, interpretable and steerable AI. anthropic.com
Stress-tests unsupervised elicitation and easy-to-hard generalization methods on three realistic challenges (salient confounds, imbalanced data, uncertain points), finding that proposed ensembling and combined approaches still fail to reliably handle them.
Anthropic analyses how often Claude is used for emotional support, advice, and companionship (2.9% of conversations), examining topics, pushback, emotional tone, and safeguards against unhealthy attachment and delusion.
Anthropic evaluates whether steering interpretable features in Claude can reliably mitigate social biases in model outputs, testing effects on quantitative evals and broader capabilities.
Tests whether RLHF-trained LLMs can morally self-correct when instructed, finding the capability emerges around 22B parameters and improves with scale and RLHF.
Anthropic's foundational RLHF paper training helpful and harmless assistants with preference models, showing alignment training costs little capability and studying reward versus KL divergence scaling.