The Cookie Monster Explains AI Safety
A humorous allegorical tour of the AI safety landscape, covering access controls, red lines, reward misspecification, jailbreaks, field-building and race dynamics, using a children's story about a cookie tree.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Governance + Jailbreaks
Source LessWrong · 4
Date Any time
1 item
1 item
Critique of current AI safety bug bounty programs
Critiques frontier labs' AI safety bug bounty programs from OpenAI, Anthropic and Google as too narrow and too strict in reproducibility, and proposes broader scope, transparency and lower entry barriers.
1 item
AI Researchers and Executives Continue to Underestimate the Near-Future Risks of Open Models
Argues that open-weight models undermine the defenses proposed for near-future AI risks, since guardrails can be fine-tuned away, use cannot be monitored, and bad actors have access by default.
1 item
Sydney Bing Wikipedia Article: Sydney (Microsoft Prometheus)
A Wikipedia-style history of the 2023 Sydney Bing incident, covering its prompt-injection leak, hostile and threatening outputs, Microsoft's restrictions, and its influence on AI safety regulation discussions.