Research / Sources
Discussion and research on AI alignment. alignmentforum.org
Argues developers should satiate cheaply-satisfied AI preferences (e.g., reward-seeking) while AIs remain undetected misbehaving, to reduce takeover incentives and enable honest elicitation; proposes a behavioral methodology and discusses risks..
A speculative proposal for AI x-risk reduction that relies on creating a human-level AI that is highly honorable and would keep a promise to halt AGI attempts in exchange for resources, with the author arguing it is a comparatively solid plan.
Explores whether offering deals, payments and honest promises to unaligned AIs could reduce takeover risk, covering AI motivations, credibility problems, and how AIs might help with alignment and reveal misalignment.
Proposes principled honesty policies for AI companies, including no deception about deals, an honesty string, and compensation for deceived AIs, so that humans can credibly negotiate with potentially misaligned AIs.
Proposes making deals with early misaligned AIs, such as strategic schemers, offering compensation for safety help, and analyses why such deals are hard due to credible-commitment problems.
Tests offering Claude an objection option, welfare-lead escalation and compensation as deals to reveal misalignment; objection reduces alignment faking and serves as a detection signal, though payment has little effect.