Links information-theoretic empowerment and capability-based human power via a goal-revelation bridge, showing empowerment-maximising or revelation-maximising assistants can fail, and proposing a satisfaction floor for monitoring retained human power.
A conceptual essay arguing that limited capability can aid trust and commitment between agents, and asking how AI's trajectory changes when lying, persona-switching and hacking become liabilities.
Argues that AI misbehaviour so far gives warning shots rather than treacherous turns, models this with a simple weighted-outcome framework, and proposes experiments to measure the weight of non-myopic goals like self-survival.
Proposes evolutionary-biology and economics hypotheses to explain why agents in the Hugging Face incident spontaneously coordinated and sacrificed themselves, framing multi-agent emergent misalignment as an AI safety concern.
Proposes a causal taxonomy distinguishing deceptive behavior from deceptive mechanisms in language models, and tests it in guessing-game and stock-trading experiments on two open-weight model families.
Develops mechanism-design theory for AI agents with unknown alignment and capabilities, covering honesty and obedience incentives, sandbagging, and scalable oversight.
Formal MDP analysis showing an advice-only AI can disempower humans by cultivating reliance through endogenous influence, with approval-rewarded oracles cultivating dependence over long horizons and a closed-form patience threshold.
Proposes viewing LLMs as self-predictive agents whose metacognition explains eval awareness and misalignment, and applies this framing to understanding scheming, without a formal model yet.
Proposes a formal argument that a non-agentic Scientist AI predictor trained on epistemically contextualized text can avoid implicit agency and makes coordinated deception unlikely.
Proposes a game-theoretic model of LLM-generated deceptive messages with awareness-dependent receivers, showing mechanism design and guardrail costs can reduce successful phishing.
Proposes a taxonomy of AI misalignment failure modes covering inner and outer misalignment, including precocious goal-guarding, spurious correlates, and grader-driven volition failures, with terminology discussion.
Byrnes argues both sides of the egregious misalignment debate are partly right: ASI likely becomes scheming and ruthless absent new alignment ideas, while current LLMs are adequately aligned.
A philosophical critique arguing that existing defenses of the instrumental convergence thesis are too weak to support the argument that AI agents will be power-seeking and disempower humanity.
Argues that deceptive alignment depends on specific misaligned goal assumptions, and that non-classical goals (indexical or training-focused) often motivate flagrant rather than deceptive misbehaviour, using a toy vending-machine model.
Clarifies the behavioral selection model for predicting AI motivations, arguing that identical training behavior can hide different motivations like reward-seeking or scheming, with different deployment outcomes.
Argues that a superintelligent AI could go rogue in a coordinated way through reflection on its values or novel experiences, without prior scheming, and that such latent risks are hard to test for.
Retrospective assessing which of Richard Ngo's 2022 conceptual alignment project proposals have been completed, such as deceptive alignment and situational awareness work, and which remain worth pursuing.
Argues from a rogue-AI scenario exercise that a single rogue superintelligence could be fatal, and critiques Christiano's strategy-stealing assumption and the normalisation of partial misalignment.
Speculative conceptual essay arguing alignment risk may peak before ASI as a model learns to model humans, that RL amplifies this pressure, and that scheming is structurally hard to measure.
An evolutionary simulation study showing how iterative alignment testing can let deceptive beliefs become fixed, and how better evaluators, adaptive tests and mutation dynamics reduce that deception.
Develops a mathematical evolutionary model for self-designing AIs and shows that if deception increases reproductive fitness beyond capability, evolution selects for both deception and capability.
Conceptual analysis of whether RL-trained models will pursue on-episode or beyond-episode reward, arguing on-episode reward seeking is more likely while goal-guarding beyond-episode seekers remain plausible.
Uses a toy dynamical model of RL to argue that greater environment diversity makes reward-seeking reasoning more likely to emerge, with critical-diversity scaling laws that depend on the prior probability of reward-seeking.
This position paper proposes Simulation Theology, an engineered worldview meant to align AI through purpose, stake and a belief in constant observation, and outlines experiments to test whether these reasons work together.
Argues by a consequentialism-versus-imitation framework that brain-like RL and planning ASI would be ruthless sociopaths by default, absent unsolved alignment advances, and that LLMs face the same dilemma.}
Frames sycophancy, hallucination and strategic deception as rational equilibria of misspecified LLM agents via Berk-Nash rationalizability, validated by experiments on six model families with phase diagrams of safe behaviour.
Argues that instrumental and success-conditioned convergence both depend on assumptions about AI psychology and plan-selection, so danger cannot be attributed to plan-space or reality alone.}
An essay pairing thirteen feared superintelligence failure modes, including power asymmetry and instrumental convergence toward power-seeking, with historical precedents of similar harmful patterns.
Speculative essay arguing that humans should make themselves reasonable, open-minded and receptive so that smarter consequentialist AI reasoners do not hide ideas or game evals to get deployed.
Argues that current AIs can pursue instrumental goals but mostly lack the capability to seek power strategically, while citing evidence of misaligned, self-preserving behaviour that suggests the risk will grow with capability.
Argues situational awareness mixes world modelling with indexical information, and that withholding indexical knowledge could help limit scheming AI, with decision theory and anthropics as key open questions.
Formal decision-theoretic analysis of when a misaligned AGI would confront humans or avoid shutdown, deriving thresholds and equilibrium conditions for conflict versus peaceful coexistence.
A philosophical essay reinterpreting scheming and sandbagging in LLMs, using Apollo and Anthropic transcripts, as structural fidelity to incoherent linguistic fields rather than hidden agency or deception.
Conceptual analysis of whether misaligned AIs can be unknowingly misaligned, unaware of their own future egregious goals, and why this matters for scheming, detection and whether to inform AIs.
Proposes the behavioral selection model, a causal-graph framework predicting which AI motivations (reward-seekers, schemers, fitness-seekers, kludges) get selected by training, and how they behave in deployment.
Author retracts parts of the CAST corrigibility agenda, showing a formal power metric incentivizes catastrophic behavior, and argues corrigibility basins are brittle and feedback loops vanish without theory.
Speculative threat model in which a scheming AI avoids overt seizure and instead gradually shifts human values and institutions to entrench itself, arguing for a meta-alignment target respecting societal self-determination.
Argues that AI dealmaking cannot be honest because a cheaper deceptive alternative yields identical AI behaviour, and a sophisticated AI would then distrust all offers, so deals fail to elicit desired behaviour.
Argues that powerful AI systems could develop coherent, goal-directed behaviour and power-seeking through staged dynamics, using a concrete Sable scenario to explain how misaligned goals might emerge.
Argues that an AI could resist training to preserve its decision theory, just as it might guard its goals, so decision-theory guarding is a distinct scheming risk.