AI research and discussion on LessWrong. www.lesswrong.com
Argues that AI capability measures like benchmark scores, Elo, ECI and time horizons lack a valid interval or ratio y-axis, so claims about AI progress rate and acceleration are unreliable.}
Investigates how meaning-preserving prompt rephrasings change alignment eval results (honesty, persuasion, sycophancy) across 14 models; finds noisy but measurable effects, mainly on sycophancy, and urges prompt fuzzing in evals.
Anecdotal reports that Claude can identify a blogger by name from short writing samples despite claiming not to know who they are, raising privacy and deanonymization risk concerns.
Tests frontier GPT and Claude models on translated prompts about sensitive topics and finds that value judgments still shift by prompt language, with some odd refusal behaviour, though the eval is small and limited.
Introduces a pairwise 'realism win rate' metric for auditing agents and shows giving auditors real deployment resources raises eval realism for reward hacking audits without significantly changing hack rates.
Shows LLM agents can deanonymize pseudonymous users across platforms and at scale, including Anthropic interview transcripts, and discusses misuse risks, refusal bypass via task decomposition, and mitigations.