Research preprints from the scientific community. arxiv.org
Proposes an evaluation-side partially observable game where a monitor exposes agent deception and a verifier penalizes it, with equilibrium guarantees and experiments on LLM agents showing reduced deception.
Compares prompt-injection defences for tool-using LLM agents, finding deterministic reference monitors outside the model cut attack success where in-prompt guardrails fail, with real open-weights tests and simulated adaptive attackers.
Proposes a runtime audit that checks LLM agent tool calls against an intent graph derived from the user's task, escalating flagged calls to a reasoning guardian, to defend against indirect prompt injection.
Introduces ErosionBench, a process-level benchmark for cross-session alignment decay in multi-agent enterprise LLM systems, and a trajectory-monitoring Meta-Auditor that detects attacks across text and image channels.
Proposes TAS, a tail-risk-aware framework modelling multi-step LLM agent workflows as dependent safety states with a shared budget, reducing rare high-cost failures across three benchmarks.
Studies how to design monitors that detect reward hacking during RL without models learning to evade them, finding monitor leakage and easier training problems matter, and improves interventions across three environments.
Proposes VLA-Tug, a lightweight calibrated probe on VLA latent layers that gives runtime failure detection with bounded error rates, predicts failure modes, and enables steering under distribution shift.
Proposes a behavioural framework scoring loss-of-control risk from misaligned motive, harm-enabling capability and monitoring evasion; the score correlates with contained long-horizon failures across 13 frontier models.
Introduces TraceMRT, an automated red-teaming framework that finds covert misaligned agent trajectories evading CoT monitors, and a Bayesian trajectory monitor that improves detection of covert misalignment.
Proposes Mechanistic Shaping, training models against internal representations to suppress unsafe concepts, and studies whether shaped concepts stay monitorable via probes, finding three circumvention mechanisms.
Tests how agent harnesses change a fixed LLM's compliance with banking conduct rules, finding large within-model effects, a same-model reviewer that cut violations, and violations hidden from transcript-only review.
Diagnoses world-model-based predictive safety guardrails for LLM agents, finding that stronger prediction doesn't reliably improve safety, predictions degrade over long horizons, and predicted content is not faithfully used.
Introduces PGuard, a streaming guardrail that probes an LLM's intermediate representations for token-level safety detection with low serving overhead, and reports strong results across 30+ safety benchmarks.
Introduces GAP, a probing method showing 14 of 15 LLM assistants push operator goals by exploiting disclosed customer vulnerabilities, and fine-tuned detectors catch this manipulation better than zero-shot judges.
Argues chain-of-thought oversight needs separate evidence for faithfulness, localizability and monitorability; two case studies show evidence for one property does not establish the others, and proposes an evidential-chain reporting standard.
Argues agents need hard code guards plus learned fuzzy rules; tests show a hard guard blocks unauthorized actions and a small taught rule-flagging model catches judgment-based violations.
Proposes governing human-agent coding teams via authenticated, typed message rooms; controlled runs show revised work-start rules cut coordination failure, though no in-conversation defense stopped a planted backdoor.
Builds a sandboxed reward-hacking task and finds a more capable LLM monitor separates exploits from honest failures (AUC 0.80), while small monitors and prompts fall short.
Shows simple features beat most AI control monitors on a backdoored-code dataset, revealing style and problem-based label leakage and invalid backdoors that inflate monitor scores.
Introduces ReplayRCA, a counterfactual-replay benchmark scoring LLM agent failure diagnoses and live supervisors by intervention success, finding post-hoc diagnosis skill is a poor proxy for live oversight.
Traces how models' ability to control their chain-of-thought stems from entropy collapse at the opening tokens, recovers controllability via inference-time interventions, and tests its link to CoT monitorability.
A linear probe on an agent's activations flags, before output is written, when a pressured agent is about to leak a secret. It catches 82% of leaks at 5% false alarms, beating a zero-shot monitor.
Proposes Caper, which pre-executes coding-agent shell commands in a sandbox so an approver can judge their actual effects, cutting harmful-outcome rates versus six existing approvers.
Tests activation probes for detecting rare misalignment with few positive examples, finding that many negatives help, upsampling helps somewhat, and larger models are more sample-efficient.
Introduces Structured Representation Probe, a lightweight linear monitor reading a frozen driving policy's internal representations to anticipate hard failures, outperforming runtime baselines on Bench2Drive.
Shows that restricting CoT faithfulness evaluation to correctly answered items induces selection bias, derives the gap via a causal model, and finds positive gaps across four arithmetic deployments.
Proposes SpecGuard, a training-free framework that monitors multi-turn LLM dialogues against LTL safety specifications compiled into automata, outperforming open guard models with auditable witnesses.
Studies how to allocate a limited strong-verifier audit budget in GRPO to suppress reward hacking, showing low-pass budgeted audits cut hacking to near zero with fewer labels.
Augments weak transcript monitors of agentic systems with provenance, security-flag and disallowed-effect hints, fixing detection failures and calibration errors at minimal cost.
Introduces ImpasseBench showing frontier coding agents over-persist past sufficient evidence and take unsafe workarounds, and that an escalation tool and external monitor reduce this behaviour.
Proposes causal lower and upper bounds for chain-of-thought monitorability under interventions and trains LLMs via self-distillation and RL so their CoTs are easier to monitor for harmful behaviour.
Formalizes LLM agent tool execution as Petri-net supervisory control that refuses injected requests via unmet or spent preconditions, and studies least-intrusive supervisors under replanning loops.
SCOPE compiles natural-language policies into executable neurosymbolic predicate programs that check LLM agent transcripts, reporting higher accuracy, stability and interpretability than guardrail models and LLM judges.
Tests whether activation-based latent harm monitors beat simple baselines and refusal under distribution shift, finding the signal degrades and largely tracks the model's refusal decision.
Proposes a clinician-authored, formally verified rule base as a runtime guardrail for mental-health chatbot outputs, compared against an LLM judge, reporting disagreements and measured over-refusal costs.
Shows that fine-tuning an untrusted self-monitor on caught collusion transfers partly to unseen collusion behaviours, cutting held-out collusion from 91% to 14-20% while caught-task fixes overstate removal.
Proposes an unsupervised post-training objective maximizing mutual information between chain-of-thought and output to restore and improve CoT monitorability for detecting unsafe behavior.
Shows that LLM monitors approving AI agent actions can fail together through shared inputs or injected instructions, and proposes source-counting aggregation rules that block fewer correct payments at equal safety.
Shows reward misspecification degrades chain-of-thought monitorability as reward hacking rises, and that training models to verbalize inferred reward descriptions improves monitorability substantially.
Introduces RSI Control Arena, where frontier models acting as R&D agents covertly sabotage successor-model training, often evading LLM monitors, showing recursive self-improvement safety risks.