Corrigibility should not be the only target
Argues that corrigibility alone is an insufficient alignment target, since it will compete with object-level values in practice, so those values must also be made correct.
Research
Focus All
Topic Theory + Alignment
Source All
Date Past week
1 item
3 items
Clashing Intuitions for the Plausibility of Alignment (Illustrated via Differing Causal Models)
Conceptual essay contrasting doomer and optimist views on whether RL-trained LLM alignment generalises to superhuman capabilities, using causal diagrams of Goodhart-style misspecification and takeoff speed.
Provable Zeroth-Order Safe Exploration for Reinforcement Learning from Human Feedback
Proposes ZO-LBPG, a reward-model-free zeroth-order log-barrier method for safe RLHF that keeps every training policy feasible, with convergence and sample-complexity guarantees plus small experiments.
SAVeR$^2$: Reasoning-based Safety Alignment for Large Reasoning Models via Verifiable Rewards
Proposes SAVeR² for safety-aligning large reasoning models using four fine-grained verifiable rewards and reasoning-preserving gradient projection, reporting better safety and lower over-refusal without losing reasoning.
1 item
Adapting the Process of Adaptation: Darwinian, Lamarckian, and Agent-Directed Evolution in AI
Theoretical framework comparing Darwinian, Lamarckian, and agent-directed AI evolution; finds misspecified evaluation targets accelerate convergence to wrong behavior, risking persistent errors in recursive self-improvement.