Thinking Machines Lab and its full-duplex interaction model

Thinking Machines Lab, the company Mira Murati launched in February 2025, has one real-time voice project in public view: an interaction model, previewed in May 2026, that listens, watches and talks at the same time. The lab has published the design and its own benchmark results.

Nov 202613 min readMarkdown

1. The May 2026 interaction-model preview

On May 11, 2026, Thinking Machines Lab posted “Interaction Models: A Scalable Approach to Human-AI Collaboration” on Connectionism, its research blog. It announced a research preview of TML-Interaction-Small, a model trained from scratch that takes in audio, video and text continuously and responds in real time. The model is a mixture of experts with 276 billion parameters, 12 billion of them active.

With it, a frontier-scale lab entered full-duplex speech, where a model listens and talks at the same time. The post positions the work against two existing approaches. Most commercial real-time speech systems, it says, rely on voice-activity detection to decide when a user’s turn has ended. Full-duplex models such as Moshi, PersonaPlex and Nemotron VoiceChat do without that component, and the lab calls them “smaller scale models focused on latency rather than intelligence benchmarks.” Its own position is that interactivity has to live inside the model, so that “scaling a model makes it smarter and a better collaborator.”

The lab repeated the argument two months later in a mission essay, The Future Worth Building Is Human (July 10, 2026), which calls interaction models a “long-term bet” and describes today’s channel between people and AI as “a small text box and a long wait.”

Most of the voice work is still closed. The interaction model has no public API, no released weights, no published price and no stated license. Inkling and Inkling-Small, the models the lab has released with open weights, accept audio but answer only in text, and its newest model, Inkling 2 Small, does not take audio at all. On voice, the public record comes down to one detailed blog post with a benchmark table, a research grant program, an inference feature contributed to the open-source SGLang serving engine, and two audio-input models with fine-tuning recipes.

Thinking Machines is betting that a large model can learn turn-taking itself, without a pipeline around it to manage turns. We think the bet deserves attention: a frontier lab has made interaction a research priority and published enough of the design for others to follow it. The weak part is the evidence. Every number comes from the lab, and outside its research preview nobody can run the model to check them.

2. Founding, people and funding

Mira Murati was OpenAI’s chief technology officer until September 2024. She launched Thinking Machines Lab on February 18, 2025, as chief executive, with former OpenAI research executive Barret Zoph as CTO and OpenAI co-founder John Schulman as chief research officer (TechCrunch later gave his title as chief scientist). SiliconANGLE counted 29 employees at launch, from OpenAI, Google, Meta, Mistral and other companies.

The lab’s home page says that instead of focusing only on fully autonomous AI, it wants to build “multimodal systems that work with people collaboratively,” and it calls multimodality critical to “more natural and efficient communication.”

On July 15, 2025, the lab said it had raised $2 billion at a $12 billion valuation. Andreessen Horowitz led the seed round, and Nvidia, Accel, ServiceNow, Cisco Ventures, AMD and Jane Street took part. The company had not yet announced a product. In her post announcing the round, Murati described multimodal AI that works “through conversation, through sight, through the messy way we collaborate.” On March 10, 2026, the lab and NVIDIA announced a multi-year partnership to deploy at least one gigawatt of NVIDIA Vera Rubin systems, with deployment targeted for early the following year. NVIDIA also made what the announcement calls “a significant investment” and did not disclose the amount.

Co-founder Andrew Tulloch left for Meta in October 2025. On January 14, 2026, Murati said the lab had “parted ways” with Zoph and named Soumith Chintala as CTO. Within the hour, OpenAI said Zoph, co-founder Luke Metz and former OpenAI staffer Sam Schoenholz were returning to OpenAI (TechCrunch).

Until the voice work, the lab’s public output was research posts and one developer product. Connectionism began on September 10, 2025 with Defeating Nondeterminism in LLM Inference, followed by posts on modular manifolds, LoRA and on-policy distillation. Tinker, launched on October 1, 2025, is an API for fine-tuning open-weight models on the lab’s own clusters. Both turn up again in the voice work. The interaction-model post uses the batch-invariant kernels from the nondeterminism post, and Inkling’s audio recipes run on Tinker.

3. How the interaction model works

Two models work together. TML-Interaction-Small stays in the conversation the whole time, and when a request needs more reasoning than it can produce on the spot, it hands the task to a background model that runs asynchronously. What it sends is “not a standalone query, but the full conversation.” Results stream back, and the interaction model works them in when that suits what the user is doing. According to the July Inkling announcement, serving as that background reasoning model was “a major goal” of Inkling’s design.

200 ms micro-turns

The model takes in 200 milliseconds of input, generates 200 milliseconds of output, and repeats. Input and output are both treated as streams rather than whole turns, and no external turn detector or dialog-management component decides who speaks. The post says the model tracks by itself whether the speaker is “thinking, yielding, self-correcting, or inviting a response,” and silence, overlap and interruption stay in its context as timed slots.

This lets the model jump in when the context calls for it instead of waiting for the user to stop. It can speak at the same time as the user, as in live translation, keeps a direct sense of elapsed time, and can search, browse or generate UI while it talks. Some of the post’s examples need vision as much as audio, such as “tell me when I’ve written a bug in my code.”

Encoder-free early fusion

There is no large standalone audio encoder of the Whisper kind, and no separate text-to-speech decoder. Audio enters as dMel, a discretised form of the mel spectrogram (Bai and colleagues, 2024), through a light embedding layer. Images are cut into 40-by-40-pixel patches and encoded with an hMLP, and speech comes out through a flow head. Every one of these parts was trained from scratch together with the transformer.

Serving and safety

Running a model in 200 ms slices means many small prefill and decode steps, each under a tight latency budget, and the post says existing inference libraries often carry too much overhead per turn for that. The lab’s answer is what it calls streaming sessions. The client sends each chunk as its own request, and the server appends it to a persistent sequence held in GPU memory. The lab says it upstreamed a version of this to SGLang, where the pull request was merged at the end of February 2026.

For safety training, the lab generated spoken refusals with a text-to-speech model, aiming for refusals that sound natural in speech without being any less firm. It also used automated red-teaming to build multi-turn refusal data for long speech conversations. The post does not name the text-to-speech model.

4. The published benchmark results

The lab compares TML-Interaction-Small with OpenAI’s GPT-realtime-2.0 and GPT-realtime-1.5, Google’s Gemini 3.1 Flash Live and Qwen 3.5 Omni-Plus-Realtime, with GPT-realtime-2.0 and Gemini each at a low and a high reasoning setting. Every number in this section comes from the lab’s post.

Interaction is measured with FD-bench. The four scenarios the post lists for version 1.5 (user interruption, user backchannel, talking to others and background speech) are those of Full-Duplex-Bench v1.5, an academic benchmark from Guan-Ting Lin and colleagues. On version 1, TML-Interaction-Small reports a turn-taking latency of 0.40 seconds, against 0.57 for Gemini 3.1 Flash Live (minimal) and 1.18 for GPT-realtime-2.0 (minimal). Its version 1.5 average is 77.8; the other configurations score between 39.0 and 54.3. On version 3, which tests tool use, it reports 82.8 percent response quality and 68.0 percent pass@1 with the background agent switched on.

The intelligence results are less one-sided. On Audio MultiChallenge, a multi-turn spoken dialogue benchmark, the model scores 43.4 percent, behind the 48.5 percent of GPT-realtime-2.0 at its xhigh setting, which is the best result in the table. The post claims only that the model is “more intelligent than any non thinking model.” On QIVD, a streaming question-answering test over video and audio, its 54.0 percent ranks sixth of the seven configurations, ahead only of GPT-realtime-1.5 at 41.2; Qwen 3.5 Omni-Plus-Realtime leads at 59.0. On text instruction following (IFEval) it scores 89.7, against 95.2 for GPT-realtime-2.0 (xhigh).

Thinking Machines also built tests for behaviour that existing benchmarks miss. Two of them are internal. TimeSpeak checks whether the model speaks at times the user asked for, such as a breathing reminder every four seconds, and CueSpeak whether it speaks at the right moment on a verbal cue; full credit on CueSpeak requires speaking while the user is still talking. Three more adapt public video datasets: RepCount-A for counting exercise repetitions aloud, ProactiveVideoQA for answering at the moment a video reveals the answer, and Charades for saying “start” and “stop” around an action. TML-Interaction-Small scores 64.7 on TimeSpeak and 81.7 on CueSpeak, against 4.3 and 2.9 for GPT-realtime-2.0 (minimal), and 33.5 on ProactiveVideoQA, where staying silent scores 25.0. Every other model the lab tried, “including thinking high models,” stayed silent or gave wrong answers on these tasks, the post says.

Caveats

These are vendor-reported results on configurations the vendor chose, and the model is not available for anyone else to rerun them. TimeSpeak and CueSpeak are internal and graded by an LLM judge, and the post does not say they will be released. The post itself says the existing interactivity benchmarks “do not adequately capture” what it describes, which is why the lab built new ones. It follows that the field has no shared yardstick yet for the behaviour the lab claims.

Of all these numbers, the FD-bench v1.5 average is the one we would most like to see reproduced. The benchmark is public and academic, and the margin is wide: 77.8 against 54.3 for the best of the other configurations. The internal tests are harder to weigh. The lab designed them around what its own architecture was built to do, and nobody outside the lab has run them.

5. Releases, grants and undisclosed details

The models the lab has released since May all listen without speaking. Inkling (July 15, 2026) is a mixture of experts with 975 billion parameters, 41 billion active, pretrained on 45 trillion tokens of text, images, audio and video and released under Apache 2.0. Its model card lists text, images and 16 kHz WAV audio as inputs and text as the only output. Audio goes in the same encoder-free way as in the interaction model, and the lab says Inkling transcribes speech, follows spoken instructions and reasons over longer recordings. Inkling-Small (July 30) has the same published size as TML-Interaction-Small, 276 billion parameters with 12 billion active, and likewise takes audio in and gives text out under Apache 2.0. Neither post says whether the two models share weights.

Developers can call Inkling through APIs from Together AI, Fireworks, Modal, Databricks and Baseten, and try Inkling-Small with audio in the Tinker Playground. Beyond hosted inference, the main audio offer is fine-tuning. The lab’s open-source Tinker Cookbook has audio recipes for Inkling and Inkling-Small covering speech recognition on LibriSpeech, speaking style and transcription on the Expresso dataset, and medical transcription of dictated prescriptions.

Inkling 2 Small (November 5, 2026) opens the second generation, with 594 billion parameters and 17 billion active. Its announcement says it “does not support audio”; it takes text and images. The lab said it intended to release the weights under Apache 2.0 the following week, after external safety testing and red-teaming. On November 10, 2026, the lab’s Hugging Face page listed Inkling and Inkling-Small but not Inkling 2 Small.

We found the missing audio the most surprising detail in the record. Inkling was designed in part as the background model for a system built around voice and vision, and the first model of the next generation takes no audio. The announcement gives no reason.

The lab also funds outside research. On May 19, 2026, it announced interactivity research grants of $100,000 each plus $25,000 in Tinker credits, with applications due by June 19. First on its list of directions was “evaluations for real-time multimodal interaction.” The others were safety for real-time multimodal systems, explaining results through generated UI, and steering agents during long tasks. The lab’s news page has not named recipients.

As of November 10, 2026, these details were missing from the lab’s pages and from the press coverage we read:

• There is no public way to use the interaction model. The May post promised a limited research preview “in the coming months” and “a wider release later this year.” SiliconANGLE reported that only a select number of partners get it during the preview, and the lab’s news and blog pages list no wider release, API or price.
• The interaction model’s weights and license are unpublished.
• Nothing is said about its training data: how many hours of conversational audio, in which languages, and how much was recorded, licensed or synthetic. Inkling’s model card names only categories, namely publicly available, acquired from third parties, and synthetic.
• The post does not list supported languages, though its examples include translating Spanish to English live.
• Larger interaction models have not appeared. The post says the lab’s larger pretrained models are “currently too slow to serve in this setting” and plans larger releases later in 2026; none had been announced by November 10.

6. Implications for full-duplex voice

Most full-duplex voice systems so far either wrap a turn-based model in voice-activity detection or use a smaller native model that listens and speaks at once. Thinking Machines has described a third kind, a large native model that also watches video.

Serving limits model size

Thinking Machines expects interactivity to improve with model size, yet the same post says its larger models are too slow to serve in 200 ms slices. For full-duplex systems, that makes inference engineering a limit on model size alongside training. The streaming sessions the lab contributed to SGLang target the overhead of serving 200 ms chunks; the post does not say whether they would help serve its larger models. If we were building a voice product, this is the trade-off we would weigh first. A bigger model may handle interaction better, but only if it can answer inside each 200 ms slice, so until the larger models can be served, the gain from scale is a promise.

Turn detection inside the model

If turn-taking moves inside the model, developers no longer tune a voice-activity detector and a dialog manager as separate parts. Citing Rich Sutton’s essay “The Bitter Lesson,” the post argues that hand-built components like these will be outpaced by general capability. Its evidence for that is its own benchmark table. We read the argument as a reasonable bet that is still unproven, and it will stay unproven until someone outside the lab can run the model.

Training data

A model that decides when to backchannel, yield or interject has to learn those behaviours from examples of overlap, interruption and timing, whether recorded or synthetic. The May post explains the architecture in detail and says nothing about the conversational audio behind it. In our view that is the largest gap in what has been published, and the detail that would tell other teams most about whether the approach can be repeated.

The open models

For developers, the open releases are an Apache 2.0 base for speech understanding that can be fine-tuned on Tinker. They do less for full-duplex voice, which needs a model that speaks. The only such model the lab has is still a preview, and the first second-generation model dropped audio input.

Things to watch next:

• A wider release of the interaction model, with an API, a price and a license.
• The larger interaction models the May post planned for later in 2026.
• Results from the grants, especially new evaluations of real-time interaction.
• Whether TimeSpeak and CueSpeak are released.
• Whether later Inkling 2 models take audio again.

Sources

• Thinking Machines Lab: Interaction Models (May 11, 2026); Interactivity Research Grants (May 19, 2026); The Future Worth Building Is Human (July 10, 2026); NVIDIA partnership (March 10, 2026); Announcing Tinker (October 1, 2025); home page.
• Model releases: Inkling and model card; Inkling-Small and model card; Inkling 2 Small and model card; Hugging Face organisation.
• Code: Tinker Cookbook audio recipes; SGLang pull request 19171.
• Press: SiliconANGLE on the launch (February 18, 2025), the seed round (July 15, 2025) and the interaction models (May 11, 2026); TechCrunch on the CTO change (January 14, 2026); Mira Murati on X (July 15, 2025).
• Papers: Full-Duplex-Bench v1, v1.5 and v3; Audio MultiChallenge; dMel.

Next: Shizuku AI: the AI VTuber company that built a Japanese full-duplex model for its character