AI safety via debate

In plain terms

If you cannot check an expert's answer yourself, you can still make two experts argue about it in front of you and judge the argument. Debate turns that courtroom intuition into an AI training protocol, with theory saying an efficient judge can get far-beyond-their-level answers right, and experiments showing weak judges really do get more accurate.

AI safety via debate is an oversight protocol in which two AI systems argue opposing answers to a question while a weaker judge, human or model, decides the winner. Proposed by Irving, Christiano, and Amodei (2018), it is the scalable oversight proposal with the strongest combination of theoretical grounding and experimental support, and as of 2026 the most active research line in the lineage.

Mechanism

A debate is a game. Given a question, one debater is assigned (or chooses) the honest answer, the other an incorrect one, and they exchange arguments, each free to point out flaws, cite evidence, and challenge the other's claims. A judge too weak to answer the question directly reads the exchange and picks a winner. The design assumption is an asymmetry: refuting a lie is easier than sustaining one, because a lie eventually contradicts something checkable, so in equilibrium the honest debater has the advantage. If that holds, training debaters to win trains them toward honesty, and the judge's verdicts become a usable training signal for questions no judge could evaluate alone.

The scheme's relationship to iterated amplification is sibling rather than rival: amplification decomposes a question cooperatively, debate decomposes it adversarially, letting the opponent choose which branch of the argument to attack. The adversarial version concentrates the judge's attention where disagreement actually lives.

Theory

The original complexity-theoretic result set the ambition: a polynomial-time judge with optimal debaters can decide problems in PSPACE, where direct judging of a single answer reaches only NP. The claim is about the protocol's ceiling, not about trained systems, but it established that verdicts from weak judges can, in principle, certify answers far beyond the judge.

Two later results strengthened the honest side's position. Doubly-efficient debate (Brown-Cohen, Irving, et al., 2023) shows a setting where the honest strategy wins with polynomially bounded work even against computationally unbounded dishonest strategies: the liar can spend forever, and refutation stays affordable. Prover-estimator debate (Brown-Cohen et al., 2025) addresses the field's best-known theoretical failure, the obfuscated arguments problem, in which a dishonest debater spreads the flaw so thinly across a long argument that no single step can be pinned as wrong; the revised protocol incentivizes honesty at comparable compute even under that strategy.

Experimental record

The evidence accumulated in three eras.

• Toy era. The 2018 paper's sparse-pixel MNIST experiment: a judge seeing only 6 pixels classified digits at 59.4 percent alone and 88.9 percent after a pixel-revealing debate between two players (48.2 to 85.2 percent with 4 pixels). Artificial, but the shape of the claim, weak judge made strong by adversarial argument, survived scaling.
• Human-judge era. Michael et al. (2023) at NYU ran human debaters on reading-comprehension questions where judges could not see the passage, finding debate helps supervise unreliable experts.
• LLM era. The landmark is Khan et al. (2024), an ICML 2024 best paper from Anthropic, UCL, and NYU: on QuALITY reading comprehension with hidden text, debate lifted non-expert LLM judges from 48 to 76 percent accuracy and human judges from 60 to 88 percent. The result inside the result: optimizing debaters for persuasiveness made judges more accurate, not less, which is the design assumption showing up in data. At larger scale, DeepMind's Kenton et al. (2024) ran nine task families and roughly five million generations, finding debate consistently beats consultancy (one model arguing its assigned side) and shows more mixed advantages over direct judging. An MIT cross-protocol comparison (Engels et al., 2025) found debate the strongest tested protocol at fixed capability gaps.

Status

The research center of gravity sits with Google DeepMind, where debate anchors the amplified oversight strand of its AGI safety approach, and with the UK AI Security Institute, where Geoffrey Irving continued the line. UK AISI published an alignment safety case sketch based on debate in 2025, the first attempt to write out what a deployment argument resting on debate would actually claim, and funds debate work through its Alignment Project. Open problems named by the field include training debaters by self-play (Arnesen et al., 2024), judge confirmation bias (2025), and the central external-validity question below.

Limitations

The main caveat is the shape of the evidence: nearly all positive results are on information-asymmetry tasks, where the judge lacks facts the debaters can reveal, rather than expertise-asymmetry tasks, where the judge lacks the concepts to follow the argument even with every fact displayed. The superhuman regime is the second kind, and whether the refutation asymmetry holds there is exactly the open question. Debate also consumes model-generated arguments, so it inherits the eliciting latent knowledge bound: knowledge no argument can surface stays outside the protocol. It assumes judge verdicts are meaningful signal, which context-sensitive behavior of the kind shown in alignment faking complicates. And it remains an oversight protocol rather than a deployed training method: no frontier model is known to be trained primarily by debate as of 2026.

Primary source in the Library

The Library's explained record of Irving, Christiano, and Amodei (2018), AI Safety via Debate, explained, covers the protocol, the complexity-theoretic argument, and the first experiments.

Related

• Scalable oversight, the parent problem and the sibling proposals.
• Weak-to-strong generalization, the complementary empirical paradigm.
• Safety case, the institutional form the debate sketch fills in.
• The survey, for debate's institutional history.

Sources

• Irving, Christiano, Amodei, AI Safety via Debate, arXiv, 2018; blog.
• Brown-Cohen, Irving et al., Scalable AI Safety via Doubly-Efficient Debate, arXiv, 2023.
• Brown-Cohen et al., Avoiding Obfuscation with Prover-Estimator Debate, arXiv, 2025.
• Michael et al., Debate Helps Supervise Unreliable Experts, arXiv, 2023.
• Khan et al., Debating with More Persuasive LLMs Leads to More Truthful Answers, arXiv, 2024.
• Kenton et al., On Scalable Oversight with Weak LLMs Judging Strong LLMs, arXiv, 2024.
• Engels et al., Scaling Laws for Scalable Oversight, arXiv, 2025.
• Arnesen et al., Training Language Models to Win Debates with Self-Play, arXiv, 2024.
• Google DeepMind, An Approach to Technical AGI Safety and Security, arXiv, 2025.
• UK AI Security Institute, An Alignment Safety Case Sketch Based on Debate, arXiv, 2025.
• UK AI Security Institute, The Alignment Project.