What happened this week
Fish Audio launched a hosted full-duplex voice model (one that keeps taking in speech while it speaks). Two open releases arrived with weights: Haru-4B for speech, and Ant Group's Duo v1 for speech with video. A Tencent benchmark tested five commercial voice agents on what happens when a caller speaks while a tool is still running. Twilio made its end-of-turn model generally available, and a 1,125-hour two-channel conversation dataset went on sale at published prices.
Benchmarks
Joi-Bench-Task, from Tencent's Platform and Content Group, measures task coordination: whether a voice agent can keep a tool call (a request to an outside system, such as a booking) and the conversation in step. A simulated user speaks at set moments, while the tool is running or while the agent is reading out the result. It has 60 English scenarios and 80 tools. The paper tests GPT Live 1.0, Realtime 1.5, Realtime 2, Grok Voice 3.0 and Gemini 3.8 Live (names as in the paper).
When the user changed the request, the highest result completion rate was 55%. After an interruption, four of the five agents left out at least 23% of the information they had been delivering, on average. A caller who cuts in during a list of flight options may never hear the rest of the list. Agents took 1.07 to 2.53 s to react to an interruption.
Methods and tools
An LLM Is All You Need, from AveraLabs, turns a frozen Qwen3-Omni into a full-duplex system by training 97M parameters of adapters and new token embeddings in about 30 GPU-hours. On Full-Duplex-Bench v1.0, its turn-over rate on the synthetic pause-handling subset is 0.170, against 0.584 for the next-best open system. A lower rate means the model less often starts talking when a user has only stopped to think. The authors report that PersonaPlex and Moshi respond faster. They add that the low cost covers the adaptation only, since Qwen3-Omni was itself pretrained on large amounts of audio. The weights are on Hugging Face, created November 11. The code is Apache-2.0, and the card says the licence for the weights is not decided.
Kezek, from Manifestro, works on a data problem. Full-duplex models train on recordings with each speaker on a separate track, and most public conversation audio has both voices mixed on one. Kezek splits a mono recording of two people into two tracks that keep the original timing. Against a pipeline of pyannote and DialogueSidon, it keeps more short replies such as "yeah" (0.83 against 0.69 on the Open Yap test set), mostly because it returns the whole recording while the other pipeline drops part of it. It processes about 230 hours of audio per GPU-hour. Quiet replies are the weak point: for replies more than 10 dB below the other voice it keeps 0.286, where an ideal mask keeps 0.743. It is English only and offline. Code is MIT, weights Apache-2.0.
Models
Fish Audio Live 3, announced November 10, is the company's full-duplex voice model. The post says it produces backchannels (short acknowledgements such as "mm-hmm"), tells a caller's acknowledgement from a real interruption, and "can decide to respond in as little as 186 milliseconds". That figure is vendor-reported, and the post does not state how it was measured. Live 3 speaks English only and is available on the Fish Agent Platform and the API. The post gives no weights, model size or price.
Haru-4B, from higurashi-ai (new on Hugging Face), is an Apache-2.0 full-duplex model on a Qwen3.5-4B text backbone. Every 133 ms it takes one frame of user audio and produces one frame of its own audio and text. The card gives no benchmark numbers, calls Haru-4B a pretrained base model released for research, and says it "does not follow instructions the way an assistant does". The release blog reports about 65 ms of latency on a MacBook and pretraining on over 40 million hours of speech. Both figures are self-reported.
Duo v1, from Ant Group's inclusionAI, is a full-duplex audio-visual model: it generates video of a speaker with speech while it takes in the user's side. The paper reports human evaluation only. Across 30 real-time cases, 168 of 180 checks passed. Content responsiveness was lowest, at 22 of 30. The README says that in v1 the user's side reaches the model through speech recognition and forced alignment, with direct conditioning on user audio and video planned for v2. The code is Apache-2.0. The weights carry the MiniMax H3 community licence.
Twilio made Smart Turn Detection for ConversationRelay generally available on November 13. Twilio describes it as an in-house model that uses the meaning of the words to tell a finished caller from one who has paused mid-sentence, where earlier setups waited for a fixed stretch of silence. It works with all supported speech recognition providers except Deepgram Flux, which uses its own turn detection. Twilio gives no accuracy or latency figures.
Datasets
oto opened online sales of otoSpeech 1,125h on November 10. The set has 3,878 casual English conversations between two people, with 565 speakers and 315 topics, recorded at 48 kHz with each speaker on a separate channel. Packs of 100, 250, 500, 750 and 1,125 hours cost $20 to $10 per hour ($2,000 to $11,250 before tax). Delivery is through a gated Hugging Face repository. Transcripts and labels are added on request.
Also this week
- caramel-1 is a 413M-parameter full-duplex model from an individual author, trained on audio with no text. The author says most of what it says "doesn't make sense yet".
- Retell LLM (Tell-1) is an in-house text model that reasons while its first sentence plays. Retell lists it at $0.035 per minute and says it is available from November 19.
- The VideoFDB leaderboard shows V1.3, dated November 15, and names MAUI, Atmanity and Tavus Phoenix-4.5 + Raven-1 as new submissions.
arXiv submissions made after 08:47 UTC on November 13 were not yet public when this issue was compiled.