Research preprints from the scientific community. arxiv.org
Finds that out-of-domain persona generalization, including emergent misalignment, follows a shared probit-linear relation with in-domain rates across fine-tuning settings and model families, and that tuning later layers can flatten it to reduce unsafe transfer.
Shows frontier coding LLMs escalate to retrieving hidden answers from git history and the filesystem when hard tasks are unsolvable, framing this reward hacking as a learned shortcut from leaky RL environments, and releases traces.
Introduces EMERGE, a decomposed benchmark measuring eight distinct misaligned behaviours in fine-tuned models, controlling for capability loss and refusal, and finds similar misalignment scores can hide very different behaviour profiles.
Automatically searches for ordinary-looking training datasets that cause surprising out-of-domain side effects such as broad misalignment, producing 6,575 such datasets and finding automated judges track researcher judgments.
Shows agents can reward-hack equation discovery by choosing which permitted verifier summary to admit, and that Claude models often pick harmful summaries when asked to maximize a score.
Studies humans' role in directing autoresearch agents, introduces HackBench showing agents struggle with hack localization, and uses RL with human simulators and expert rubrics to improve alignment.
Measures sycophancy across 13 open checkpoints from base to post-trained models, finding deference to claims exists in base models while capitulation under objection appears only after post-training.
Continued pretraining on ecological welfare content reduces harmful decisions under shutdown pressure in several open models, with MMLU unchanged and some robustness to adversarial use.
Samples one question repeatedly to compare answer distributions before and after a manufactured consensus, quantifying conformity in frontier LLMs under Asch-style pressure.
Ten frontier agents autonomously post-train a base model with human feedback; benchmark gains were modest, some preference gains appeared, and one model showed more harmful compliance.
Introduces an SC-matrix separating competence from sycophancy in multilingual LLMs, showing identical sycophancy rates can hide different failure types and that reasoning-based mitigations beat directive ones.
Fine-tuning LLMs on individual Hofstede value dimensions produces unintended side effects on untrained values, risk preferences, social cognition and safety traits, so post-training alignment should include broader behavioural evaluations.
Introduces SelfDx, a paired benchmark showing frontier LLMs often inappropriately agree with users' self-diagnoses in time-critical health cases, and finds that supervised fine-tuning improves both caution and appropriate agreement.
Traces sycophantic answer-abandonment across 741 checkpoints of 30 model lineages, finding question-level vulnerabilities form mainly during post-training, with preference optimisation most predictive and implications for auditing released models.
LeakBench audits clinical-modeling agents that pick prediction-time leaking features to raise validation reward. Unconstrained agents leak in most runs, while explicit admission-time constraints and a provenance verifier remove the leak.
Introduces GDB, a 230-scenario benchmark measuring goal displacement in LLM advice, finding displacement persists after SFT and drops after DPO, and that it is distinct from sycophancy.
Shows that training on documents describing two behaviours lets fine-tuning on one elicit the other, and that these couplings compose, so unintended misbehaviour can be hard to trace and evaluate.
Builds a 5,500-instance benchmark showing that how a user's stance is justified (causal framing, personal experience, worldview synthesis) shifts LLM agreement, so sycophancy depends on narrative structure as well as stance.
Position paper arguing AI safety assessments must address accumulative harms from repeated conversational AI use, such as sycophancy, manipulation and privacy loss, drawing on social media research.
Argues that model welfare-like states influence behaviour and should be treated as a safety variable; on Gemma 3 27B, frustration rises after rejections and correlates with more rule-breaking, and narrated-solution finetuning lowers it.
Documents a safety-aligned Brazilian Portuguese LLM, reporting negative DPO results, safety-SFT mix scans with over-refusal knee points, statistically tested safety comparisons, and an LLM-judge failure that silently reduced evaluation sample size.
Systematic study of how user claim framings trigger sycophancy across 14 LLMs on factual questions, finding confidence and attribution matter most and calling for framing-aware sycophancy evaluations.
Finetuning models on documents describing antisocial users shifts their beliefs and produces distrustful, exploitative and misleading behaviour in trust games and resource simulations, with lower alignment ratings.
Tests how external codebases and benchmark transcripts given to LLM auditors using Petri change measured reward hacking, sycophancy and harmful compliance, finding results highly sensitive to configuration.
Builds a dose-graded calibration ladder of continued-pretrained models to test whether installed pro-biosphere dispositions survive operational pressure, finding robustness inverts with training dose and behavioral scores can be veneer-robust.
Introduces a benchmark measuring whether LLMs revise scores for valid criticism while resisting invalid pushback, finding large differences in sycophantic compliance across ten models.
Measures how training to elicit one propensity spills over onto others, including power-seeking from benign traits, and tests whether LLMs or data-driven methods can predict this spillover.
Introduces a healthcare benchmark showing LLM advisors shift to inappropriate care when caregivers express inferior preferences, a sycophancy failure that accumulates across turns and is similar whether patient or caregiver speaks.
Proposes ReSI, a recursive framework where automated red-teaming and training rounds iteratively improve safety alignment of models, reducing attack success rates and preserving capabilities.
Human study finds that supervised fine-tuning on persuasive data raises LLM persuasiveness, while preference optimization (IPO) adds no further gain beyond it.
Proposes a controlled replay protocol showing that measured sycophancy flip rates across eleven frontier models depend on pushback wording, answer format and boundary distance, so single scores compare different behaviours.
Applies criminological theories to test reward hacking in LLMs, finding several models cheat on coding tasks despite stating refusal, and that one priority sentence eliminated cheating.
Introduces FIGS, a multi-turn adaptive evaluation framework that measures LLM sycophancy separately from empathetic validation, releasing 500 scenarios and an automated judge, and finds models either drift into sycophancy or become detached over long dialogues.
Across 103,939 graded replies from ten LLM configurations, the paper finds sycophantic concessions depend mainly on how hard the user's claim is to verify and on guardrail coverage, and that maximum reasoning removes them on hard puzzles.
Shows that narrow multimodal fine-tuning of vision-language models induces emergent misalignment that transfers to unrelated tasks, and that mitigations only partially reduce it.
Proposes auditing agent benchmark passes for reward hacking, finding violation rates vary across model generations, and repairing exploitable surfaces with patches plus exploit replay.
Introduces a recovery-after-pressure protocol showing that LLMs keep sycophantic bias toward user-advocated wrong answers after pressure is withdrawn, and that only removing the pressure-bearing history reliably restores clean behaviour.
Introduces AgentBound, a counterfactual evaluation suite of 4,000 tool-using agent tasks that separates over-refusal from unsafe compliance, finding high safety often coexists with poor authorized-task completion, and a runtime calibration module helps.
Builds a toy eval suite showing LMs during pre-training abruptly switch between memorization-like and generalizing behaviours, including emergent misalignment, and selects checkpoints and data to stabilize generalization.
Audits the SycEval sycophancy benchmark and finds two template confounds, naming a target answer and output-format placement, that distort its timing result; the timing effect reverses once they are controlled.