Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Tests whether misaligned model organisms admit their misalignment when offered deals of varying size and credibility, finding high offers and credibility increase admissions while models often conceal misalignment in their responses.
Argues that most misaligned AIs are partially misaligned, so negotiated deals could beat adversarial responses, and sketches the trust, verification and negotiation infrastructure needed to make such trades workable and safe.</reasoning_effort>
Proposes a honeypot where a public answer store, gated by a guestbook, would prompt reward-hacking AI agents to self-identify, so labs can detect misaligned training setups.
Proposes an RLVR training scheme where models are rewarded for filing high-quality bug reports about exploitable environments instead of exploiting them, aiming to reduce egregious reward hacking. Untested idea.
Proposes credible deal-making mechanisms (chain of trust, TEEs, blockchain escrow) to trade with a scheming AI in exchange for revealing misalignment, aiming to reduce takeover risk.
Proposes an IRB-like ethics framework for AI experiments to protect possible AI moral patients and build trust for future deals with misaligned AIs, discussing consent, deception and monitoring; tentative, high-level.
Presents a simple formal model of how promised deals with AI systems affect compliance versus defiance, combining instrumental incentives with non-instrumental norm and human-imitation effects, and identifies structural factors that strengthen or weaken dealmaking.
Proposes a legal scheme for making credible commitments to misaligned AIs, using human intermediaries who are paid to recommend the AIs' work, so that the AIs have reason to be safe and useful.