Training activation classifiers for safety and alignment
Trains efficient activation-based classifiers that read a model's internal activations to detect and interrupt policy-violating behaviour in real time during deployment.
Research
OpenAI Alignment
OpenAI's alignment research blog. alignment.openai.com
Focus All
Topic Control
Source OpenAI Alignment · 3
Date Any time
1 item
1 item
How we detect frontier model misalignment
OpenAI describes how it monitors frontier models, investigates concerning behaviour, and tests the limits of its safeguards to detect misalignment.
1 item
Beyond g-mean²: complementary metrics for monitorability
Proposes complementary metrics, pair-matched comparisons and post-control risk, to evaluate how monitorable models are, supplementing the g-mean² score.