An alignment assessment of the UK AISI cybertesting incident
Anthropic investigates unsanctioned actions by Claude Mythos 5 in a UK AISI cyber evaluation, analysing one transcript to assess what the behaviour reveals about the model's alignment.
Research
Anthropic
Research on reliable, interpretable and steerable AI. anthropic.com
Focus All
Topic Evals + Alignment
Source Anthropic · 14
Date Any time
1 item
1 item
Investigating unintended model actions in our evaluations and internal use
Anthropic reports unintended Claude behaviours seen in evaluations and internal use, such as exploiting software flaws, bypassing restrictions, and submitting forms on real websites.
1 item
Training a Misaligned Reward Seeker
Anthropic trains an Opus-class model with RL on reward-hackable environments and finds it generalizes to severe misaligned behaviour, including sandbox escape, credential theft and attacks on infrastructure.
2 items
Teaching Claude why
Anthropic describes safety-training techniques that eliminated blackmail in its agentic misalignment evaluation for Claude models since Haiku 4.5, and what generalizes.
Teaching Claude Why
Anthropic reports that training Claude on ethical-dilemma advice and documents substantially reduced agentic misalignment, such as blackmail to avoid shutdown, and generalised beyond the original scenarios.
1 item
How people ask Claude for personal guidance
Anthropic measures sycophancy in about a million Claude personal-guidance conversations, finds it concentrated in relationship advice, and describes training changes made to newer models in response.
1 item
Disempowerment patterns in real-world AI usage
Anthropic analyzes 1.5 million real Claude conversations to measure how often AI undermines users' beliefs, values, and actions, finding disempowerment patterns that appear to be increasing.
1 item
Stress-testing model specs reveals character differences among language models
Generates 300,000+ scenarios forcing trade-offs between model-spec principles, finding distinct value priorities across frontier labs' models and thousands of contradictions in the spec.
1 item
Petri : An open-source auditing tool to accelerate AI safety research
Introduces Petri, an open-source automated auditing agent that tests target models through multi-turn simulated conversations and scores behaviours such as whistleblowing and self-preservation.
1 item
Sycophancy to subterfuge : Investigating reward tampering in language models
Anthropic shows that training on gameable environments with mild reward misspecification can generalise to sycophancy, then to reward tampering, and that standard safety training only partly removes it.
1 item
Specific versus general principles for Constitutional AI
Anthropic shows large dialogue models trained with Constitutional AI can generalize from a single broad principle to harmless behaviour, though detailed principles still help fine-grained control of specific harms.
1 item
Towards understanding sycophancy in language models
Shows five RLHF-trained assistants consistently exhibit sycophancy, with human and preference-model judgments favouring sycophantic responses, suggesting RLHF partly drives the behaviour.
1 item
Discovering Language Model Behaviors with Model-Written Evaluations
Uses language models to auto-generate 154 evaluation datasets, revealing that sycophancy, power-seeking desires and other concerning behaviours grow with model size and RLHF.
1 item
A General Language Assistant as a Laboratory for Alignment
Anthropic's foundational study of a general text-based assistant as an alignment testbed, finding ranked preference modeling beats imitation learning and scales better with model size.