Full-duplex voice AI is evaluated on how it behaves in a conversation: when it takes the turn, whether it backs off when interrupted, how it handles pauses and short acknowledgements, and how fast it responds. Newer benchmarks also check whether a voice agent gets the task done.
What gets measured
Full-Duplex-Bench
Full-Duplex-Bench, from researchers at NTU, UC Berkeley, UW and MIT, is the most cited family of full-duplex benchmarks. Much of its scoring rests on one rule: the system has taken the floor if it says more than three words or speaks for more than a second.
- v1 (2025) scores turn-taking, backchannels, interruptions and pause handling on fixed recordings.
- v1.5 adds overlap: a user interruption, a listener backchannel, a side conversation or background speech while the system talks.
- v2 streams audio to the system under test in real time and adds multi-turn dialogue.
- v3 (2026) tests multi-step tool calls driven by real recorded speech with disfluencies such as self-corrections.
Why scores differ
The same model can score differently on the same benchmark depending on who ran it. The benchmark's authors run it one way, a model's own team may run it again with small changes, and a third party may change the rules, for example a longer takeover threshold or a different judge model. Scores from different runs are not comparable.
Analytics shows each run on its own tab and says who ran it. Read more: Why STS needs new benchmarks
Conversation versus task
Full-Duplex-Bench and TurnBench score conversational behaviour. Agent benchmarks score whether the call achieves its goal. τ-Voice, part of τ-bench, runs live full-duplex calls under noisy, accented conditions, and models that score about 85% on the text version drop to 26 to 38% once voice and real time are added. EVA-Bench runs bot-to-bot calls and reports accuracy and experience as separate scores.
For the turn-end decision on its own, see Turn detection.
Full-duplex benchmarks
All 27 benchmarks in the database that test full-duplex behaviour, most recently updated first. Models scored counts the models in this database that list the benchmark.
| Benchmark | Setting | Main metrics | Models scored | Updated |
|---|---|---|---|---|
| TurnBench Sesame AI Labs / Mundo AI / CMU / oto | Lab | end-of-turn recall, interruption recall, false-positive rate | 3 | 2026-11 |
| Duplex-MPE Ma et al. | Lab | response initiation, answer accuracy, silence preservation | 0 | 2026-11 |
| InteractionBench Avera Labs (He et al.) | Live | conversational timing, spoken-task accuracy, whisper and volume control | 0 | 2026-11 |
| VideoFDB NVIDIA / David AI | Live | perception rubric (0 to 5), generation rubric (0 to 5), TOR-Alignment | 4 | 2026-10 |
| DuplexAct Dong et al. | Lab | immediate execution success, follow-through success | 8 | 2026-10 |
| MultiPartyTurnBench Mathur and Manocha | Lab | end-of-turn recall at fixed FPR, interruption recall at fixed FPR, signed latency | 2 | 2026-10 |
| M3-DuplexBench NTT (Fukuda et al.) | Lab | turn-taking rate and latency, pause handling, backchannel and barge-in response | 0 | 2026-10 |
| DuplexLedger Li et al. | Lab | latest-value accuracy, final-state fact coverage, stale assertions | 0 | 2026-10 |
| MP-Bench Shih et al. | Lab | turn-taking accuracy, response appropriateness, comprehension accuracy | 0 | 2026-10 |
| τ-bench 2.0 (τ-Voice) Sierra Research | Vertical | pass^k, tool-use success, voice task success | 3 | 2026-10 |
| Audio MultiChallenge Scale AI (Scale Labs) | Lab | Inference Memory, Instruction Retention, Self Coherence | 2 | 2026-10 |
| MultiTalkBench Wang et al. (MultiTalk) | Lab | long-range entity tracking, topic coherence, addressee selection | 0 | 2026-09 |
| ECHO Zhao et al. | Lab | pair accuracy, yield accuracy, keep accuracy | 0 | 2026-09 |
| DuplexSpeechBench-IFEval Mathur and Manocha | Lab | Instruction Adherence Score, Persona Adherence Score | 0 | 2026-09 |
| APEX-Voice Mathur and Manocha | Vertical | Pass@1, Reliable@3, artifact field accuracy | 0 | 2026-09 |
| Full-Duplex-Bench v3 NTU (NVIDIA advisory) | Live | Pass@1, tool-call F1, response latency | 2 | 2026-05 |
| MTR-DuplexBench Zhang et al. | Lab | conversational features, dialogue quality, instruction following | 0 | 2026-04 |
| SID-Bench Xia et al. | Lab | Average Penalty Time (APT), false alarms, response delay | 0 | 2026-03 |
| Full-Duplex-Bench v2 NTU / UC Berkeley / UW / MIT | Live | streaming latency, multi-turn coherence, overlap handling | 0 | 2026-02 |
| HumDial (ICASSP '26) ICASSP 2026 Spoken Dialogue Challenge | Lab | turn-taking, paralinguistic grounding, human preference | 0 | 2026-01 |
| FLEXI Ge et al. | Lab | latency, response quality, emergency interruption | 0 | 2025-09 |
| MMedFD Academic (MMedFD authors) | Vertical | duplex WER, medical-term recall, overlap-region WER | 0 | 2025-09 |
| Full-Duplex-Bench v1.5 NTU / UC Berkeley / UW / MIT | Lab | overlap duration, mutual-silence rate, interruption recovery | 3 | 2025-08 |
| FD-Bench Peng et al. | Lab | interruption response, delay, robustness under noise | 0 | 2025-07 |
| J-Moshi (subjective) Ohashi et al. | Lab | naturalness, meaningfulness | 0 | 2025-06 |
| Talking Turns Apple / CMU | Lab | turn-event F1, backchannel rate, interruption rate | 3 | 2025-04 |
| Full-Duplex-Bench v1 NTU / UC Berkeley / UW / MIT | Lab | turn-taking latency, back-channel timing, interruption success | 5 | 2025-03 |
Read more: Mapping the benchmark landscape
FAQ
- Which benchmark should I start with?
- Full-Duplex-Bench is the most cited family. Versions 1 and 1.5 score fixed recordings, and versions 2 and 3 stream audio to a live system. For turn ends on their own, use TurnBench. For whether a voice agent completes a task, use τ-Voice or EVA-Bench.
- Can I trust a vendor's own numbers?
- Treat them as a starting point. A vendor chooses the test set and settings for its own run. A benchmark run by its authors, or by a third party under the same protocol, is easier to compare. The database says who ran each number.
- Is there a Japanese full-duplex benchmark?
- 2 of the 27 full-duplex benchmarks in the database cover Japanese: M3-DuplexBench, J-Moshi (subjective).