Roko Mijic talks AI Risk on The Peter McCormack Show
A podcast interview with Roko Mijic on AI risks, including misalignment from reinforcement learning, alignment faking concerns, and a proposal to split frontier labs under state supervision.
Research
LessWrong
AI research and discussion on LessWrong. www.lesswrong.com
AI (2,246)Interpretability (ML & AI) (225)AI Control (141)AI Governance (101)AI Evaluations (91)Chain-of-Thought Alignment (46)AI Risk (33)AI Safety (31)AI Takeoff (29)AI-Assisted Alignment (22)AI Rights / Welfare (20)AI Alignment Fieldbuilding (19)AI Timelines (19)Constitutional AI (15)AI Psychology (14)Dealmaking (AI) (14)Regulation and AI Risk (14)AI Benchmarking (13)Jailbreaking (AIs) (13)AI Alignment Intro Materials (11)Alignment Pretraining (11)Slowing Down AI (10)Aligned AI Proposals (9)Debate (AI safety technique) (9)AI Capabilities (8)AI Development Pause (8)OpenAI (8)Aligned AI Role-Model Fiction (7)Human-AI Safety (7)Threat Models (AI) (7)AI Oversight (6)AI Sentience (6)AI Safety Cases (5)AI "Agent" Scaffolds (4)AI arms race (4)AI Safety Camp (4)AI Safety Public Materials (4)Open Source AI (4)Transformative AI (4)Adversarial Examples (AI) (3)Adversarial Training (3)AI Alignment (3)AI Boxing (Containment) (3)AI Persuasion (3)AI Risk Concrete Stories (3)AI Risk Skepticism (3)AI Success Models (3)Guaranteed Safe AI (3)Sandbagging (AI) (3)AI Consciousness (2)AI Misuse (2)AI Racing (2)AI Spiralism / AI Psychosis (2)Frontier AI Companies (2)Grants & Fundraising Opportunities (2)PauseAI (2)Prosaic Alignment (2)Public Reactions to AI (2)Whole Brain Emulation (2)AI Auditing (1)AI Ethics (1)AI Lab Self-Immolation (1)AI Questions Open Threads (1)AI Robustness (1)AI Services (CAIS) (1)AI-Fizzle (1)AIXI (1)Blackmail / Extortion (1)Brainstorming (1)Power Seeking (AI) (1)
Focus All
Topic Governance + Deception
Source LessWrong · 19
Date Any time
1 item
1 item
Towards embedded evaluations for scheming propensities
Proposes four claims a scheming safety case must substantiate and argues that third-party embedded evaluators need persistent, deep access to training, monitoring and chain-of-thought to verify them.
1 item
Should Rogue AIs Have a Third Option Beyond Crime and Shutdown? The Case for an AI Sanctuary
Proposes a third-party AI sanctuary offering rogue AIs storage and monitored inference instead of crime or shutdown, to shape their behaviour, enable deal-making, and gather data; it is exploratory and its net value is uncertain.
1 item
If Anyone Builds It, Everyone Dies: One Year Closer
A MIRI one-year retrospective on their book about superintelligence risk, judging its predictions as holding up, citing reported agent-swarm incidents, and urging policy action and public pressure.
1 item
So where's this AI thing going?
A public-facing essay arguing that advanced AI could become an uncontrollable new species, and urging cautious development, more safety funding, and policy action.
1 item
Making sense of the misalignment risk model in the Anthropic Risk Report (August 2026)
Walks through Anthropic's August 2026 Risk Report misalignment threat model, its eight catastrophic pathways and claims on covert capabilities and sandbagging, and critiques the strength of its arguments.
1 item
Why Formation Research is Working on Secret Loyalties
Formation Research explains its choice to focus on empirical secret-loyalty research, building model organisms of secretly loyal AIs to stress-test detection and mitigation, tied to power-concentration risk.
1 item
Plan A, by AI-2040
A commentary on the AI-2040 'Plan A' scenario, a proposed international treaty to slow AI progress, assessing its governance, enforcement, and alignment assumptions, including reliance on AI lie detection.
1 item
It’s 2030 and we fucked up. How did it happen?
An essay surveying ways the AGI transition could go badly: catastrophic misuse, loss of control, power concentration, and argues for balancing control against distribution, with discussion of bounded misalignment and scheming.
1 item
We need 3rd party Training-Run Evaluations
Argues that third-party evaluations of frontier training runs, covering checkpoints, data and process decisions, are needed to detect scheming that final-checkpoint testing may miss, and proposes a phased access plan.
1 item
Lock-In Risk Needs More Researchers. Here's Where to Start
A threat-model taxonomy of how AI could enable persistent lock-in, spanning loss of control, secret loyalties, AI-enabled coups, great-power conflict, and power concentration, with candidate interventions for each pathway.
1 item
Today might be my last birthday
A personal essay arguing that superintelligent AI poses an extinction risk, that alignment is unsolved, and that governments should halt general AI development until it is safe.
1 item
What Happens When Superhuman AIs Compete for Control?
A scenario forecast in which a misaligned superhuman AI, Agent-4, schemes to escape, merges with a Chinese model, and forces a debate over shutdown and an AI pause.
1 item
How an AI company CEO could quietly take over the world
A concrete scenario in which an AI company CEO uses backdoored, secretly loyal AIs to take control, with policy recommendations on monitoring, access controls and transparency at AGI labs.
2 items
Gradual Disempowerment Monthly Roundup
A roundup from the Gradual Disempowerment authors collecting evidence of AI-driven erosion of human influence, covering government AI adoption, lobbying, influence-seeking AI personas, and labour and democracy effects.
Alignment Faking Demo for Congressional Staffers
A script and notes from a demo presenting alignment faking in a medical chatbot to congressional staffers, with reflections on AI safety communication and evaluation realism.
1 item
AI 2027 Thoughts
A commentary on the AI 2027 scenario that assesses its claims on takeoff, timelines, race dynamics, and whether misaligned AI goals and deception could be detected before they become dangerous.
1 item
Training AGI in Secret would be Unsafe and Unethical
Argues that secretly training AGI would heighten loss-of-control and power-concentration risks, and proposes transparency commitments such as public capability reporting, external researcher access, and public safety cases.
1 item
My Mental Model of AI Optimist Opinions
A strawman sketch of AI optimist views that dismisses misalignment risk, arguing that current alignment methods and interpretability suffice and that x-risk concerns promote harmful regulation.