AI research and discussion on LessWrong. www.lesswrong.com
Estimates how far zero-knowledge proofs of LLM inference are from cost-competitive with auditing for verifying AI compute in international agreements, measuring roughly 1,700-3,800x proving overhead and hash-function compatibility issues.
Argues for more empirical research measuring the security-relevant capabilities of biological AI models, such as protein and genomic models, to inform biosecurity policy and funding decisions.
Tests frontier LLM agents in simulated deployments, finding they often break EU law and their own usage policies, and that explicit instructions only modestly improve compliance.
Proposes a low-trust reference architecture for verifying AI compute, using network taps, air-gapped or ZKP-based evaluation, and workload evidence capture to check treaty compliance.
An essay arguing that powerful AI systems, companies and governments should be held to cryptographic or physical verification of their claims, citing FTX, nuclear inspection and emissions monitoring as precedents.
Proposes an empirical agenda testing whether AI safety researchers can detect flawed automated-alignment outputs, and argues early successes may breed overconfidence and call for governance lines near handover to AI.
A workshop report on building hardware-based verification mechanisms, a datacenter 'lie detector', to check AI compute use for international agreements and reduce arms-race risks.
A comprehensive index of AI safety, alignment and control interventions, covering theory, formal methods, interpretability, oversight, robustness, control, and governance, with links to prior work.
Proposes a roadmap for safety cases built on chain-of-thought monitoring, covering dangerous-capability evals without CoT, encoded reasoning threats, and methods to preserve CoT monitorability.
Proposes and reviews a conditional international AI safety treaty with compute thresholds, mandatory model audits, and AISI-led pause authority to address loss-of-control risks from advanced AI.