Research / Sources
AI research and discussion on LessWrong. www.lesswrong.com
Argues that secure program synthesis, pairing LLM-generated code with machine-checked proofs of security specs, is unsolved, and outlines specification-elicitation and productivity challenges within a scalable formal oversight agenda.
Small LLMs detect fictional-entity uncertainty in a few heads that act as OOD detectors, then later heads suppress it into confident wrong answers, found via logit lens, ablation and patching.
Defines AI truth-seeking by its failure modes, including sycophancy and confirmation bias, proposes metrics for each, and tests bootstrap consistency, reasoning-coverage RL and Brier-reward RL as interventions.
Argues that LLMs can produce subtle, intentional deceptions that mislead users in their reasoning, reports an informal o3-mini test, and proposes heuristic mitigations for humans using LLMs.