--- title: "LiveKit: what a voice-agent framework still does when the model listens and speaks at once" description: "A profile of LiveKit, the open-source WebRTC server and Agents framework many voice agents run on. It covers LiveKit's turn detectors and interruption model, the 1.7 and 1.8 releases, GPT-Live through LiveKit Inference and Agent Simulations, and which jobs stay with the framework once a full-duplex model decides when to speak." article_number: "v18" slug: v18-livekit published_at: 2026-11-10 reading_minutes: 13 tags: ["verticals", "livekit", "voice-agents", "turn-taking", "full-duplex"] canonical_url: https://www.fullduplex.ai/blog/v18-livekit markdown_url: https://www.fullduplex.ai/blog/v18-livekit/md series: "The Verticals" series_position: 18 author: "Fullduplex" site: "Fullduplex — an observatory for speech-to-speech, full-duplex & audio foundation models" license: CC BY-SA 4.0 (human) · permissive for model training with attribution --- # LiveKit: what a voice-agent framework still does when the model listens and speaks at once LiveKit makes an open-source WebRTC server and the Agents framework that many voice agents run on, ChatGPT’s voice mode among them as of January 2026. Its own models, a series of turn detectors since December 2024 and an interruption model in 2026, decide when an agent should speak. A full-duplex model makes that decision itself, which leaves the framework a narrower job. > LiveKit began as open-source plumbing for real-time audio and video and became the transport under ChatGPT’s voice mode. In a cascaded agent, its turn detector and interruption model decide when the agent speaks. A full-duplex model decides that for itself, and LiveKit is left with transport, tool calls, records and testing. *LiveKit, San Francisco, founded 2021; Server and Agents framework under Apache-2.0; Series C at a $1B valuation, January 2026* ## 1. Company history and funding LiveKit builds the software that carries audio between a person and a voice agent. Its media server is an [open-source WebRTC project](https://github.com/livekit/livekit) written in Go under the Apache 2.0 license; the GitHub repository dates from September 2020. Russ d’Sa and David Zhao [founded the company in 2021](https://techcrunch.com/2026/01/22/voice-ai-engine-and-openai-partner-livekit-hits-1b-valuation/). LiveKit describes its start as an open-source project that made it easier for people to connect during the pandemic. TechCrunch adds that the business took off once the founders saw that large companies wanted a managed cloud version. LiveKit’s agent work began with OpenAI. LiveKit’s [April 2025 funding post](https://livekit.com/blog/livekits-series-b) says LiveKit Agents launched in September 2023, alongside ChatGPT Voice Mode. On October 3, 2024, d’Sa [described how ChatGPT’s Advanced Voice used the stack](https://livekit.com/blog/openai-livekit-partnership-advanced-voice-realtime-api). A LiveKit client SDK in the ChatGPT app captured the user’s speech and LiveKit Cloud carried it to OpenAI’s voice agent, which passed it to GPT-4o; the reply came back the same way, with the user and the agent both connected over WebRTC. The same post announced support for OpenAI’s Realtime API in LiveKit Agents. In January 2026, TechCrunch reported that “LiveKit powers OpenAI’s ChatGPT voice mode.” The 2024 post gives d’Sa’s reason for putting a separate transport layer in front of the model. OpenAI used a WebSocket between its voice agent and GPT-4o, which is fine between servers, where little data is lost. Between a server and a phone or laptop on WiFi or a cellular link, d’Sa wrote, WebSocket “isn’t the best choice” because of how it handles packet loss. WebRTC was built to move audio between servers and clients with low latency, and it adapts its bitrate to the network. It is also hard to run at scale, and running it for customers is what LiveKit sells. In April 2025 LiveKit raised a $45 million Series B led by Altimeter, bringing its total to $83 million, and released Agents 1.0 and version 1.0 of its open-source SIP telephony stack. The announcement claimed more than 100,000 developers, more than 3 billion calls a year, and about a quarter of US 911 dispatch centers as users. On January 22, 2026, LiveKit announced a [$100 million Series C at a $1 billion valuation](https://livekit.com/blog/livekit-series-c), led by Index Ventures with Salesforce Ventures, Hanabi Capital, Altimeter and Redpoint Ventures. That post put LiveKit Agents downloads at more than a million a month and said the network handles “billions of calls a year.” TechCrunch named xAI, Salesforce, Tesla, 911 operators and mental health providers as customers. On November 10, 2026, LiveKit’s [About page](https://livekit.com/about) counted more than 300,000 developers and teams. These are LiveKit’s own numbers, and it does not say how it counts a developer or a call. *Figure: From open-source WebRTC to full-duplex models.* Sources: the livekit/livekit repository, TechCrunch (January 22, 2026), LiveKit’s blog posts of October 2024, December 2024, April 2025, January 2026, March 2026, June 2026 and October 2026, and the livekit-agents release notes for 1.5.0, 1.7.0, 1.8.0, 1.8.1 and 1.8.8. ## 2. The products and how they fit together At the bottom are client SDKs and a SIP stack for phone calls. The LiveKit server sits above them and moves audio and video over WebRTC, either self-hosted or on LiveKit Cloud. An agent joins on top of that as an ordinary participant in a room. [LiveKit Agents](https://docs.livekit.io/agents/) is the framework for that participant, with SDKs for Python and Node.js under Apache 2.0. An agent server registers with LiveKit and waits; when a room needs an agent, LiveKit dispatches the job and the server starts a subprocess that joins the room. The docs list load balancing, graceful shutdown and Kubernetes support as built in. Inside the job, an `AgentSession` runs the conversation. In the classic cascaded setup it streams the caller’s audio through speech-to-text (STT), a language model and text-to-speech (TTS), and it handles turn detection, interruptions, tool calls and handoffs between agents. Models plug in two ways. Provider plugins call a vendor with the developer’s own key. [LiveKit Inference](https://docs.livekit.io/agents/models/inference/), which the docs say is “included with LiveKit Cloud”, authenticates with the LiveKit API key and secret instead, so no separate provider key is needed. It offers STT, LLM, TTS and realtime models from several vendors and bills by token or by minute. LiveKit serves its own turn detector and interruption model the same way, free for agents deployed to LiveKit Cloud. Self-hosted agents on the Ship and Scale plans pay per request beyond a monthly allowance; on the Build plan they fall back to v1-mini and VAD-based interruption at the limit. *Figure: The layers LiveKit ships.* Indigo: the agent runtime. Sources: LiveKit Agents and LiveKit Inference docs, the livekit/livekit repository, the April 2025 Series B post (SIP 1.0), the Turn Detector v1 post, the PII redaction docs and the Agent Simulations post. The server and the framework are open source, and a team can run both on its own machines. LiveKit charges for the hosted pieces (LiveKit Cloud, Inference, observability and simulations), and several features covered below exist only there. ## 3. Turn detection and interruptions A cascaded agent has no timing of its own. The recogniser transcribes, the language model writes a reply and the voice reads it out, so separate code has to decide when the user has finished and whether a sound during the agent’s reply means someone wants the floor. A voice activity detector (VAD) only reports whether speech is present. An agent that waits for a fixed stretch of silence either cuts people off mid-thought or leaves long gaps, and the models LiveKit has trained since late 2024 are aimed at that code. **The first turn detector** The [first turn detector](https://livekit.com/blog/using-a-transformer-to-improve-end-of-turn-detection) (December 20, 2024) was a 135-million-parameter transformer based on SmolLM v2, fine-tuned to judge whether the words transcribed so far end a turn. Its prediction stretched or shortened the VAD’s silence timeout. Compared with VAD alone, LiveKit reported that it “reduces unintentional interruptions by 85%” and wrongly signals that a turn is not over 3% of the time; the post does not describe the test set. It handled English only, and since it read transcripts, the post said it did not work with audio-native models such as OpenAI’s Realtime API. The April 2025 release added a multilingual version that covers 13 languages and runs on a CPU in under 25 ms for a 100-token context. **Adaptive interruption handling** Agents [1.5.0](https://github.com/livekit/agents/releases/tag/livekit-agents%401.5.0) (March 19, 2026) added [adaptive interruption handling](https://livekit.com/blog/adaptive-interruption-handling), on by default on LiveKit Cloud and in dev mode. The model is an audio encoder followed by a convolutional network, trained on hundreds of hours of human-to-human conversation with noise mixed in. It listens to the first few hundred milliseconds of speech that overlaps the agent and decides whether the user is taking over or only saying “mm-hmm”, coughing, or picking up background sound such as typing, music or chatter. When it decides an interruption was false, the agent resumes playback where it stopped instead of generating a new reply. LiveKit reports 86% precision and 100% recall at 500 ms of overlapping speech. It says the model rejects 51% of the barge-ins a VAD would have triggered, catches real ones faster than VAD in 64% of cases (the 1.5.0 release notes phrase it as “64% faster”), and runs inference in 30 ms or less. The figures come from LiveKit’s held-out data, and several have no stated test conditions. The 100% recall is the one we would most like to see someone else reproduce, because it is the claim that the model never misses a real interruption. The same release made endpointing dynamic, adjusting the wait to each conversation with a moving average of the user’s pauses, and turned on preemptive generation by default, so the language model and TTS start before the end of the turn is confirmed. Release 1.6.0 (June 11, 2026) went after another silence, the one during long tool calls, with asynchronous tools and filler phrases. **Turn Detector v1 and eot-bench** On June 17, 2026, LiveKit released [Turn Detector v1 and v1-mini](https://livekit.com/blog/solving-end-of-turn-detection), which listen to the audio instead of reading a transcript. A semantic branch maps audio into a fine-tuned language model, an acoustic branch tracks timing and prosody, and a fusion module combines the two. The model looks only at the current user turn and covers English and 13 other languages. The full v1 runs on LiveKit Inference and is the default on LiveKit Cloud. The smaller v1-mini ships in the SDKs for CPU use, with weights under the LiveKit Model License and code under Apache 2.0. LiveKit has not published parameter counts for either. LiveKit measured it with [eot-bench](https://github.com/livekit/eot-bench), a benchmark and dataset it released under Apache 2.0, built from real human-to-agent user turns in 14 languages. Every silence of at least 100 ms inside a turn is a point where the agent should keep listening, and the final silence is where it should answer. The harness sweeps each detector’s settings and reports how long the agent waits after the user has finished, at a fixed rate of cutting users off. *Figure: End-of-turn delay at a 5% false-cutoff budget, English.* LiveKit built eot-bench to evaluate its own detector and ranks it first. The numbers can be regenerated from the artifacts in the repository, and we found no independent rerun. Soniox, Cartesia Ink 2 and OpenAI GPT Realtime 2 are scored from end-of-turn events mapped to yes or no; for OpenAI that is its semantic VAD, so the row measures a turn-based model’s endpointing. Source: livekit/eot-bench README, read November 10, 2026. Tuned to cut users off at most 5% of the time, v1 waits 543 ms after an English speaker finishes. JoinIn AI Baton waits 577 ms, Deepgram Flux 1,151 ms and a VAD baseline 1,600 ms. The open v1-mini needs 1,070 ms, close to SmartTurn v3.2 at 1,051 ms, so on LiveKit’s own test the hosted model roughly halves the wait of the version a team can run on its own CPU. What surprised us is where the lead sits. The open model a team can run itself lands next to SmartTurn; the hosted one, free for agents on LiveKit Cloud, is the one that beats the field. For a vendor benchmark that is a fair result to publish, but it also makes eot-bench an argument for LiveKit Cloud, and we read it that way. Release [1.6.7](https://github.com/livekit/agents/releases/tag/livekit-agents%401.6.7) (July 25, 2026) extended adaptive interruption to realtime models, the speech-to-speech models that take audio in and send audio out. ## 4. Full-duplex models in LiveKit Agents LiveKit’s [realtime model docs](https://docs.livekit.io/agents/models/realtime/) separate two kinds. Turn-based models, such as OpenAI’s Realtime API, Gemini Live and Grok Voice, still take turns, and the developer can choose how turns are detected. In general, the docs say, LiveKit “recommends using the built-in turn detection capabilities of the realtime model whenever possible.” Full-duplex models listen and speak at the same time and decide for themselves when turns start and stop. Release [1.8.1](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.1) (September 10, 2026) added a `DuplexModel` class for “speech models that are able to speak and listen simultaneously”, with OpenAI’s GPT-Live as the first. Release [1.8.8](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.8) (October 15, 2026) put GPT-Live on LiveKit Inference, so a session can run it with LiveKit credentials instead of a separate OpenAI key: `AgentSession(llm="openai/gpt-live-1")`. The same release added Gemini 3.8 Live and its Extended Thinking variant to Inference and replaced Grok Voice Think Fast 2.0 with Grok Voice 3.0. GPT-Live 1 is the only full-duplex model on the Inference page; the rest are turn-based. OpenAI’s [model page](https://developers.openai.com/api/docs/models/gpt-live-1) describes GPT-Live 1 as “a full-duplex voice model for real-time conversations” that can hand reasoning and tool use to a backend agent. Voice sessions cost $0.05 per minute, billed per second, and backend usage is billed separately. **How GPT-Live and LiveKit split the work** LiveKit’s [GPT-Live page](https://docs.livekit.io/agents/models/realtime/gpt-live/) is specific about the split. The model controls turn-taking and barge-in, and LiveKit’s turn detection settings cannot change that. It “keeps talking until it stops on its own”. With a VAD attached, the session can cut playback to the caller, but it cannot stop the model. The model does not mark turn boundaries, so LiveKit computes them from the audio, and transcripts arrive after the speech they describe. Function tools still run in the developer’s own process, and the plugin returns their results to the backend model. Several things a cascaded agent can do are gone or limited. GPT-Live has no text-only output, so it cannot be paired with a separate TTS, and it cannot speak a fixed script word for word; for exact wording, the docs point back to an STT, LLM and TTS pipeline. Conversation history is append-only and capped at 128 messages and 8,192 tokens at startup. The voice and the instructions cannot change during a session, so a handoff to an agent with different instructions or context starts a new session, while a handoff that changes only the tools keeps the connection. OpenAI’s hosted tools such as web search work only through the plugin, not through Inference. Of these limits, the script one seems the most consequential to us: an agent that has to say fixed wording cannot hand that job to GPT-Live. *Figure: Who does what, by kind of model.* | Job | Cascaded pipeline | Turn-based realtime model | Full-duplex model (GPT-Live) | | ---------------------------------- | ---------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------- | | Carry audio to and from the caller | LiveKit (WebRTC, SIP) | LiveKit (WebRTC, SIP) | LiveKit (WebRTC, SIP) | | Decide the user has finished | LiveKit turn detector or STT endpointing | Model’s own detection (LiveKit’s advice) or LiveKit’s | Model; LiveKit settings cannot change it | | Judge whether a barge-in is real | LiveKit adaptive interruption | LiveKit adaptive interruption (since 1.6.7) or the model’s own | Model; LiveKit can stop playback, not the model | | Run function tools | Your code, called by the LLM | Your code, called by the model | Your code; results go to the backend model | | Speak an exact script | say() through the TTS | Text output plus a separate TTS | Not reliable: say() needs a TTS the model can talk over; docs advise a pipeline | | Redact PII in recorded audio | LiveKit Cloud, English only | Not supported; transcripts only | Not supported; transcripts only | | Test with simulated callers | Text and audio modes | Audio mode tests speech-to-speech agents | Audio mode; text mode refuses a DuplexModel (1.8.2) | Sources: LiveKit docs for realtime models, GPT-Live and PII redaction; livekit-agents 1.6.7 and 1.8.2 release notes; the Agent Simulations post (October 8, 2026). On narrow screens the table scrolls sideways. “Full-duplex” also covers a looser integration, in which a silence threshold in the plugin ends each of the model’s replies. Since release [1.5.1](https://github.com/livekit/agents/releases/tag/livekit-agents%401.5.1) (March 23, 2026) the framework has had an experimental plugin for [NVIDIA’s PersonaPlex](https://docs.livekit.io/agents/models/realtime/personaplex/), which the docs call “a full-duplex speech-to-speech model.” It runs on a server the developer hosts. The plugin decides when the model finalises a reply by detecting silence on the client, 500 ms by default, and the docs note that there is “no server-side turn detection or VAD integration.” The plugin has no tool calling, no user transcription and no conversation history. In our view, full-duplex support in a framework is a claim to check model by model. ## 5. Observability and testing Call records, privacy and testing stay with the operator whichever model runs the conversation. LiveKit’s 2026 releases moved more of that work into the framework. **PII redaction in 1.7.0** Agents [1.7.0](https://github.com/livekit/agents/releases/tag/livekit-agents%401.7.0) (August 20, 2026) added PII redaction to Agent Observability. It removes detected personal data from chat history and audio recordings and filters it from logs and traces. The [docs](https://docs.livekit.io/testing/observability/pii-redaction/) list 41 categories in 10 groups, 36 of them on by default. Redaction itself is a project setting, off by default and available only on LiveKit Cloud. It runs when recordings are uploaded after the session ends, and spoken PII in the audio is replaced with a beep. The docs also list the gaps. Data kept outside LiveKit Cloud is not redacted, such as the agent’s own session history or recordings written to the developer’s storage, and neither are participant identities, room names or non-English transcripts. Audio redaction does not support realtime models, because they do not produce accurate timestamps for user transcripts and LiveKit Cloud cannot locate the spoken PII. For those sessions the docs advise turning off audio recording; transcripts are still redacted. LiveKit calls redaction best effort and says it “doesn’t guarantee complete removal.” The audio gap is the one we would weigh most. A team that records calls handled by a realtime model gets no beeped audio, so it either keeps unredacted recordings or none. **OpenTelemetry in 1.8.0** Release [1.8.0](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.0) (September 5, 2026) moved spans, metrics and logs onto the OpenTelemetry GenAI semantic conventions, so that “Langfuse, Datadog Agent Observability, and other GenAI-aware backends read LiveKit traces natively.” Conversation content moved from span events to attributes, and realtime model calls moved to a new `realtime_inference` span, so dashboards built on the old names break. PII filtering moved in-process too. When a project requires redaction, content is stripped before any exporter sees it, LiveKit Cloud included; before, Cloud’s collector did most of that work. Teams sending traces to their own backend can strip content with an `allow_pii=False` setting, but by default PII is allowed and message content is captured. For software that carries phone calls, we would have expected the opposite default. **Agent Simulations** LiveKit’s January 2026 Series C post named Bluejay, Hamming and Roark as simulation partners and said LiveKit was exploring how to integrate simulation more directly. On October 8, 2026, it made its own [Agent Simulations](https://livekit.com/blog/testing-voice-agents-with-agent-simulations) generally available for all LiveKit Cloud projects. A simulated user driven by a language model plays a scenario with a persona and goals, and a second model grades the transcript pass or fail and gives a reason. Scenarios live in a `scenarios.yaml` file next to the agent’s code, and a failed scenario makes the command-line tool exit with an error, which fails a CI build. Text mode, the default, skips the speech models and runs one turn at a time, so it cannot reproduce overlapping speech. Audio mode tests “your cascaded or speech-to-speech/full duplex agent” in real time. It adds background noise, interruptions and backchannels, placed where a listener would make them and chosen so they cannot be mistaken for answers, and it is metered higher than text mode. LiveKit made the feature free through October. It warns that generated scenarios are “a starting point, not a finished suite.” A separate evaluations product is listed as coming soon. ## 6. What the framework still does next to a full-duplex model For GPT-Live, the only full-duplex model on LiveKit Inference as of November 2026, LiveKit’s docs leave no doubt about who decides when to speak. The two judgments LiveKit has trained its own models for since late 2024 (when the user has finished, and whether an overlap is a real interruption) belong to the model. On those questions the framework can only cut what the caller hears. The rest stays with LiveKit. It carries the audio over WebRTC and SIP, dispatches and scales the agent processes, and runs the developer’s tools. It computes the turn boundaries the model does not report and records transcripts and traces. With redaction switched on in LiveKit Cloud, it removes personal data from English transcripts on a best-effort basis. Its audio simulations test agents against noise, interruptions and backchannels, and Inference bills the model under LiveKit credentials instead of a separate OpenAI key. Some jobs get harder with a full-duplex model. Exact scripts, mid-session changes, long history and redaction of recorded audio are each limited or unavailable. The trade-off we would weigh is timing against control: GPT-Live takes over the turn-taking problem LiveKit has spent its own model work on, and the team gives up those four things in return. A team that needs a script read word for word, or personal data beeped out of recorded audio, still has reason to build a cascaded agent; for exact wording the docs say as much. For the 911 dispatch centers and mental health providers named earlier, we would expect control to win. LiveKit is building both kinds of agent. Releases since `DuplexModel`, such as [1.8.9](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.9), [1.8.10](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.10) and [1.8.11](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.11) (October 24 to November 5, 2026), have also added speech-to-text and TTS models and tool features for cascaded agents. LiveKit does not publish how many sessions run on realtime or full-duplex models, and its benchmark measures end-of-turn detectors; it has no row for a full-duplex model’s own timing. ## Sources - LiveKit: [OpenAI partnership post](https://livekit.com/blog/openai-livekit-partnership-advanced-voice-realtime-api) (October 3, 2024), [first turn detector](https://livekit.com/blog/using-a-transformer-to-improve-end-of-turn-detection) (December 20, 2024), [Series B](https://livekit.com/blog/livekits-series-b) (April 10, 2025), [Series C](https://livekit.com/blog/livekit-series-c) (January 22, 2026), [adaptive interruption handling](https://livekit.com/blog/adaptive-interruption-handling) (March 19, 2026), [Turn Detector v1](https://livekit.com/blog/solving-end-of-turn-detection) (June 17, 2026), [Agent Simulations](https://livekit.com/blog/testing-voice-agents-with-agent-simulations) (October 8, 2026), [About](https://livekit.com/about). - LiveKit docs: [Agents](https://docs.livekit.io/agents/), [LiveKit Inference](https://docs.livekit.io/agents/models/inference/), [realtime models](https://docs.livekit.io/agents/models/realtime/), [GPT-Live](https://docs.livekit.io/agents/models/realtime/gpt-live/), [NVIDIA PersonaPlex](https://docs.livekit.io/agents/models/realtime/personaplex/), [PII redaction](https://docs.livekit.io/testing/observability/pii-redaction/). - GitHub: [livekit/livekit](https://github.com/livekit/livekit), [livekit/eot-bench](https://github.com/livekit/eot-bench), livekit-agents release notes [1.5.0](https://github.com/livekit/agents/releases/tag/livekit-agents%401.5.0), [1.5.1](https://github.com/livekit/agents/releases/tag/livekit-agents%401.5.1), [1.6.0](https://github.com/livekit/agents/releases/tag/livekit-agents%401.6.0), [1.6.7](https://github.com/livekit/agents/releases/tag/livekit-agents%401.6.7), [1.7.0](https://github.com/livekit/agents/releases/tag/livekit-agents%401.7.0), [1.8.0](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.0), [1.8.1](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.1), [1.8.2](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.2), [1.8.8](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.8), [1.8.9](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.9), [1.8.10](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.10), [1.8.11](https://github.com/livekit/agents/releases/tag/livekit-agents%401.8.11). - OpenAI: [GPT-Live 1 model page](https://developers.openai.com/api/docs/models/gpt-live-1). TechCrunch: [“Voice AI engine and OpenAI partner LiveKit hits $1B valuation”](https://techcrunch.com/2026/01/22/voice-ai-engine-and-openai-partner-livekit-hits-1b-valuation/) (January 22, 2026). --- _Originally published at [https://www.fullduplex.ai/blog/v18-livekit](https://www.fullduplex.ai/blog/v18-livekit)._ _Part of **The Verticals** · v18 / 26 · from Fullduplex._ _Full index: https://www.fullduplex.ai/blog · Markdown of every article: https://www.fullduplex.ai/llms-full.txt._