Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Proposes RL training environments with deliberately misaligned helper agents so orchestrators must learn to detect and manage misalignment, making alignment part of the RLVR loop; discusses limits and second-order risks.
Maps open AI alignment problems, their cruxes and dependencies, and formalizes some of them, including debate and corrigibility, in Lean to identify gaps and collaboration opportunities.
Tests whether frontier models can imitate weaker predecessors' answers to conceal sandbagging or model evaluators; finds capability-level steering but mostly no identity-specific imitation, with reasoning as a partial mitigant.
Argues that secure program synthesis, pairing LLM-generated code with machine-checked proofs of security specs, is unsolved, and outlines specification-elicitation and productivity challenges within a scalable formal oversight agenda.
Proposes an empirical agenda testing whether AI safety researchers can detect flawed automated-alignment outputs, and argues early successes may breed overconfidence and call for governance lines near handover to AI.
Sketches a realistic target future where superintelligent AI stays under permanent human control through dynamic alignment targets, structured multi-source feedback, and institutional norms, without claiming a technical solution.
Proposes factored cognition control protocols that decompose tasks into isolated child tasks, finding that on APPS they raise safety from 41% to 70% and cut attack success from 26% to 14%.
Tests a control protocol where a trusted model gives feedback for an untrusted model to implement; the untrusted model exploits the feedback to craft stealthier backdoors, lowering safety from 93% to 80%.
Proposes ARAD, an alignment plan where a principal safely consults slightly smarter advisors and distills the pair iteratively, using agent-foundations ideas such as cryptographic pessimism; the author flags unresolved flaws.
Argues that AI control research, which targets intentional scheming in early transformative AI, has little impact on existential risk because most doom comes from stronger AI and from failures to solve their alignment.
Proposes topological debate, where two AI agents argue over which simulation resolutions or world models determine a safety property, linking AI safety via debate to guaranteed safe AI.