What happened this week
The foundational papers and the agent-SDK releases pulled in two related directions: how should a full-duplex model carry actions and tool calls on the same clock as audio, and how should the realtime stack expose policy knobs for interruption and long-user-speech without breaking the user experience. StepFun anchors the foundational side with two papers; LiveKit ships the application-layer counterparts.
Foundational — full-duplex carries actions, not just audio
DuplexSLA (StepFun + Peking University + NTU) is the paper to read this week. It proposes a native Speech-Language-Action backbone that decodes assistant audio together with a rate-limited textual action stream on a shared 160 ms chunk timeline, so that listening, speaking, planning, and tool calling all unfold on one clock. Two capabilities define the model: semantic-driven turn-taking control where interruption, pause, and backchannel are handled inside the same backbone instead of by an external semantic VAD; and in-conversation planning and tool calling, where planning text and structured tool calls are emitted on the action channel. The argument is that existing duplex backbones still hand off agentic behaviour to an external cascade — DuplexSLA pulls it inside.
StepAudio 2.5 Technical Report is the matching unified audio-LM report from StepFun. The premise is that once text and audio share a multimodal representational space, task specialisation between ASR, TTS, and realtime spoken interaction becomes a matter of data construction, optimisation targets, and decoding constraints rather than separate architectures. The post-training paradigm moves from standard supervised learning to task-tailored RLHF as the primary mechanism, and the resulting backbone matches or exceeds specialised systems on all three capabilities. The cloud product, StepAudio 2.5 Realtime, is also live this week via the StepFun realtime WebSocket API.
Foundational — synchronisation, safety, and an open audio foundation
Synchronization and Turn-Taking in Full-Duplex Speech Dialogue Models simulates full-duplex dialogues between two instances of Moshi under controlled channel-noise and decoding-bias conditions, measures internal synchronisation with Centered Kernel Alignment across temporal lags, and probes anticipatory turn-taking cues from delayed internal activations using causal LSTM probes. Strong representational synchronisation peaks near zero lag in clean conditions and degrades with noise, and internal states encode anticipatory information that supports turn-taking prediction ahead of time. It is a probing study, not a new model — but it puts a concrete number on something the field has only described qualitatively.
Stable Audio 3 from Stability AI is a family of fast latent diffusion models for variable-length audio generation and editing, with inpainting for targeted edits and continuation of short recordings. Latent diffusion runs on top of a novel semantic-acoustic autoencoder, adversarial post-training reduces inference steps while improving fidelity, and small / medium weights are released. Generation runs under 2 s on an H200 and a few seconds on a MacBook Pro M4 — the first open audio-foundation drop of this generation that targets consumer hardware.
Acoustic Interference from CUHK and HKUST is the safety counterpart. The paper shows that LALM safety alignment can be compromised purely by Acoustic Latent Semantics — paralinguistic features intrinsic to the priors of audio generative models — rather than by content injection. A set of universal, instruction-neutral interference audio decouples the attack payload from the audio signal itself and serves as a universal jailbreak trigger across standard malicious text queries. For anyone deploying audio LLMs in production, this is the threat-model paper to read.
Application layer — LiveKit Agents 1.5.10 and 1.5.12
livekit-agents 1.5.10 (May 18) ships a fix that cancels realtime generation when speech is interrupted (closing a gap on the OpenAI Realtime path), an improved should_discard check inside barge-in handling, Speechmatics added as an inference STT provider with VAD support, Rime coda model support, an inference LLM hot-swap option, and a Deepgram TTS websocket receive-timeout. livekit-agents 1.5.12 (May 21) is the more substantive release: UserTurnLimitOptions lets the session interrupt long user speech instead of waiting indefinitely (a long-standing UX gap on outbound or noisy calls), the mcp_servers param is deprecated on Agent and AgentSession in favour of a different MCP integration path, AvatarMetrics adds join-latency and playback-latency telemetry, gemini-3.5-flash is exposed as a model string, and the Perplexity Responses API LLM ships as a plugin.
What is not here
No in-window dataset drop and no reclassification surfaced with a primary source the validator can verify. Google announced Gemini Omni Flash at I/O on May 19 — primarily a multimodal video generation model with synchronised spatial audio — but it is adjacent to scope rather than inside it, and is deliberately left out. Cartesia, Hume, Deepgram (Voice Agent), and ElevenLabs Agents shipped no in-window technical changelog items.
Corrections to hello@fullduplex.ai.