Shizuku AI: the AI VTuber company that built a Japanese full-duplex model for its character

On November 8, 2026, Shizuku AI, the a16z-backed startup behind the AI VTuber Shizuku, announced Shizuku-Duplex: a Japanese full-duplex model that listens and speaks on an 80 ms clock, hands web search to a background model, and picks its character’s motion from the same network. So far the public record is a blog post, call recordings and a benchmark the company ran itself.

Nov 202612 min readMarkdown

1. The Shizuku-Duplex announcement

On November 8, 2026, Shizuku AI published the first post on its tech blog, “Shizuku-Duplex: Japanese Full-Duplex Model for Natural Conversation”, in English and Japanese. The same day, the company’s X account announced it as “the world’s first full-duplex audio synthesis model built specifically for AI characters.” The blog post claims something else. Setting aside OpenAI’s GPT-Live, it calls Shizuku-Duplex “the first full-duplex model capable of intelligent dialogue in Japanese.”

A full-duplex model listens and speaks at the same time, the way people do on a phone call. It can say “mm-hmm” while the user is talking, reply before the user has quite stopped, and go quiet when the user cuts in. Most voice assistants still run a cascade, where speech recognition, a language model and speech synthesis take turns once the system decides the user has finished. The Shizuku post sets the two designs side by side and shares five call recordings, covering backchannels, replies that do not wait for silence, stopping when interrupted, talking on through a user’s “mm-hmm,” and looking something up mid-conversation.

The company runs Shizuku, an AI character who streams on YouTube and also appears on X and Discord. Its X profile gives the aim as “Creating the world’s most lovable AI companion.” Shizuku-Duplex was built for her. Besides audio and text, the model has an output that picks a Live2D motion for the avatar while it speaks, which a voice-only assistant would have no use for.

Apart from the post, its recordings and a two-minute video on X, the only evidence for the model is a Japanese benchmark the company ran, so every benchmark number below is vendor-reported.

2. The founder and the company

Akio Kodaira, Shizuku AI’s founder and CEO (The Bridge), launched Shizuku as an AI VTuber on YouTube in January 2023, while he was still completing his PhD at UC Berkeley. According to the a16z announcement, she chatted with viewers in Japanese and English, sang songs, and answered in real time through a Live2D avatar. Kodaira ran dozens of streams and built up thousands of followers. The channel description still says she reacts to comments and talks.

His research background is real-time image and video generation. He is the lead author of StreamDiffusion, a pipeline for real-time interactive image generation first posted to arXiv in December 2023 and published at ICCV 2025. Its abstract reports up to 91.07 frames per second for image-to-image generation on a single RTX 4090. a16z adds that he worked on real-time video generation at Meta and then held a research role at Luma AI. Two other StreamDiffusion authors’ names, Kohei Ohno and Soichi Sugano, turn up again in the Shizuku-Duplex credits.

On February 9, 2026, a16z announced that it was leading Shizuku AI’s seed round, in a post by Justine Moore, Marco Mascorro and Martin Casado, without giving the amount. The post describes the company as building “an AI lab in Japan dedicated to developing AI companions and characters,” starting with Shizuku herself: “multilingual voice synthesis, richer conversational AI, and presence across platforms like Discord, YouTube, and X.” The problem a16z points to is that companions lack “the data to generate truly engaging, proactive conversations.”

The careers page, last updated July 13, 2026, describes the company as developing and operating Shizuku, “an AI companion designed to be part of everyday life.” It lists five Japan-based roles (engineering manager, software engineer, ML engineer, research engineer, and corporate and business operations) and names offices in Tokyo and San Francisco. The Japanese job descriptions say more about the work. Software engineers wire LLM, TTS and ASR models into the product. The ML engineer listing asks for low-latency streaming inference, and the research engineer role takes in speech synthesis and spoken dialogue along with other modalities (our reading of the Japanese).

Three days before the duplex announcement, on November 5, 2026, the company’s technology team posted a prototype on X in which Shizuku holds a voice conversation entirely on an iPhone. Speech recognition, a small language model and speech synthesis all run on the device, which makes it a cascade, the design the duplex post argues against. ASCII, quoting the team’s post, reported on November 9 that the phone-sized language model makes her less sharp than usual.

Ten people are credited on Shizuku-Duplex. The four core contributors are Ayuto Tsutsumi, Kohei Ohno, Yuu Jinnai and Akio Kodaira; the other contributors are Ningdong Wang, Naoki Kiryu, Hao Wang, Kazuma Fujioka, Soichi Sugano and Mizuki Sango. Three of the names carry a note saying they were members of Shizuku AI at the time of development.

3. How Shizuku-Duplex works

The post explains the design well enough to follow, though not to reproduce; it names no base model and links no code.

The 80 ms tick

Shizuku-Duplex counts time in ticks of 80 ms. On every tick it takes in the user’s audio and produces its own, speaking or silent, so it hears the user the whole time. It writes what it is about to say as text slightly ahead of the audio (the post calls this its inner monologue) and decides for itself when to start talking and when to stay quiet. All of that has to finish inside the 80 ms, which the team says took TensorRT and Triton kernel optimizations.

The step and the term both echo Kyutai’s Moshi, which the post cites when it introduces full-duplex models. Moshi’s Mimi codec works at 12.5 frames per second, an 80 ms frame (Moshi README), and Moshi predicts time-aligned text ahead of its audio under the name “Inner Monologue.” Shizuku AI calls its own architecture novel.

Interaction and background models

The post’s reasoning is that one model struggles to look things up or work through a hard request while also keeping a conversation going, so the system uses two. An interaction model handles the conversation in real time, and a background model works asynchronously on a harness with tools such as web search. When the interaction model needs help it emits a special DELEGATE token, keeps talking while the background model works, and puts the result into its own words once it arrives. In the post’s example, a user wants a Kyoto trip for two within 100,000 yen; the model says it will look for a few plans, the web search takes 7.1 seconds, and the model then reports that lodging looks like the main cost.

The post credits the split to earlier work: KAME (ICASSP 2026) and Thinking Machines Lab’s Interaction Models (May 11, 2026), which pairs a time-aware interaction model with an asynchronous background model and is the basis for the post’s own diagram. OpenAI’s GPT-Live, introduced on July 8, 2026, also hands search and reasoning to a separate model, GPT-5.5 at launch, while the conversation continues. So Shizuku AI claims a first only for Japanese. To its knowledge, Shizuku-Duplex is the first Japanese full-duplex model that can call tools, apart from GPT-Live, and the Japanese edition of the post notes that an English full-duplex model with tool calling, NemotronLabs VoiceChat, is already public.

Text, speech and motion outputs

At the core is a Transformer built on an LLM. User audio enters through a streaming audio encoder as continuous features, a choice the post says was inspired by SALM-Duplex, and is combined with the model’s own text and audio tokens from the previous tick. Results from the background model are inserted into the same sequence, a method the post notes is also used in DuplexOmni. On the output side, a text head writes the inner monologue and a local Transformer predicts audio tokens that an audio decoder turns into a waveform. The third, an action head built for the character, selects a motion “to better communicate the agent’s feeling and intent while speaking.”

Compaction

A full-duplex model’s context keeps growing even through silence, and the post cites two 2026 papers reporting that responses degrade as conversations get longer. Shizuku-Duplex summarizes the conversation roughly every five minutes and rebuilds its context from the system prompt, the summary and the turns just before the switch. The switch fits inside a single tick, so the conversation does not pause. The post does not measure how much this helps, and of all its claims this is the one we would most want to see tested. A companion is meant for long, everyday conversation, which is exactly where the papers the post cites say full-duplex models break down.

4. The company’s Japanese benchmark

Public benchmarks for Japanese full-duplex models are thin. M3-DuplexBench and Full-Duplex-Bench-JA have been published, the post notes, but their Japanese evaluation data is not public. The team therefore built its own, taking the scenarios and metrics of the English Full-Duplex-Bench (v1 and v1.5) for turn-taking, plus chit-chat and question-answering tasks from M3-DuplexBench. As a check, it built an English version the same way, ran GPT-Live on it, and reports that the results matched OpenAI’s official Full-Duplex-Bench scores.

Six models were tested, four of them full-duplex: Shizuku-Duplex, OpenAI’s GPT-Live-1, and the two Japanese academic models J-Moshi-ext and LLM-jp-Moshi-v1. The other two, GPT-Realtime-2.1 and Gemini 3.8 Live, are not full-duplex and serve as reference points. They were called through their APIs with default settings, and both reply after detecting the end of the user’s speech. The models’ output audio was transcribed with Haas-ja-v1.0, and Claude Opus 5.5 scored the content as a judge. The Japanese edition adds that these were the main models available in Japanese, not all of them.

On turn-taking, Shizuku-Duplex has the best result among the four full-duplex models on five of the six rows. GPT-Live-1 wins the sixth, the turn end, where it starts early less often (97% clean against 90%). The two models that are not full-duplex never start early and always stop when interrupted, but they also go quiet at almost every user backchannel (0% and 13% kept talking), which is the weakness of the turn-based design that the post describes. None of the six handles speech that is not meant for it well. Shizuku-Duplex, the best of the full-duplex group there, stayed out of the conversation only 17% of the time when the user spoke to someone else, and 25% of the time with background speech.

What the model says is weaker than how it times it. On every content and knowledge row Shizuku-Duplex is second among the full-duplex models, behind GPT-Live-1, though far ahead of the two Moshi-based Japanese models: 79% on multi-turn QA against 1% for both, and 6.5 against 3.2 and 2.7 on chit-chat relevance. The two models that are not full-duplex score above it on most of these rows, with 96% and 98% on multi-turn QA. Web lookup is closer, at 43% against 50% for GPT-Live-1, 43% for Gemini 3.8 Live and 15% for GPT-Realtime-2.1. For both Shizuku-Duplex and GPT-Live-1 it is also the weakest accuracy row, at roughly half their no-lookup scores (88% and 95%). In our view that matters more than the ranking: on this benchmark, answering through a background model is still the least proven part of the design, for OpenAI as much as for Shizuku AI. The post’s first figure adds two reply-quality rows for the full-duplex models. After the user finishes a turn, Shizuku-Duplex’s replies were rated 2.6 out of 5 against 4.0 for GPT-Live-1; after an interruption, 82% of its replies stayed on topic against 84%.

The post’s summary says Shizuku-Duplex showed the highest conversational naturalness of the six models. That is hard to square with its own figure, whose row of almost the same name, chit-chat naturalness scored 1 to 10 by the judge, puts Shizuku-Duplex fourth of the six at 6.2, behind GPT-Live-1 (7.6), Gemini 3.8 Live (7.7) and GPT-Realtime-2.1 (6.8). The post does not say which measurements the summary rests on.

The median time from the end of the user’s speech to the reply was 0.57 seconds for Shizuku-Duplex and 0.53 seconds for GPT-Live-1. Gemini 3.8 Live took 1.27 seconds and GPT-Realtime-2.1 took 1.61 seconds. The two pairs were not measured the same way. Shizuku-Duplex and GPT-Live were timed from server-side recordings, which leave out network round-trip time, and the two API models from local playback timestamps, which include it.

The vendor built this benchmark, chose the comparison set, ran every model and picked the judge. Matching OpenAI’s published GPT-Live scores in English is a fair sign the method works, and the post deserves credit for printing the rows it loses, the 2.6 against 4.0 reply score among them. Still, no one outside the company has run Shizuku-Duplex, and the post does not say whether the Japanese benchmark will be released.

5. Related models in this series

Japanese full-duplex speech has come up in this series before, from academic groups. The NII profile covered J-Moshi from Nagoya University and LLM-jp-Moshi-v1 from NII, both fine-tunes of Kyutai’s Moshi with open weights: J-Moshi under CC BY-NC 4.0 and LLM-jp-Moshi-v1 under Apache 2.0, released on February 25, 2026. Shizuku-Duplex is the first Japanese full-duplex model from a startup in the series, and it is closed. The company describes the architecture as its own, and in its benchmark the two academic models are the baselines it beats on content by a wide margin.

Its nearest relative in design is GPT-Live: one model that listens and speaks at once, with a background model for search and reasoning. Their stated purposes differ. The Shizuku post says GPT-Live “is designed to be a tool, a smart assistant that helps solve the problems of the users,” and that its own goal is for an AI companion “to be someone you can spend your daily life with.” OpenAI’s launch post presents GPT-Live as the model behind a more natural ChatGPT Voice that can hand deeper work to frontier models.

The Kyutai profile covers the other close neighbour. Kyutai supplied the template that both academic models fine-tune. Shizuku-Duplex shares Moshi’s 80 ms step and its text-ahead-of-audio stream, but it takes user audio in as continuous encoder features and adds delegation and an action head. Kyutai, NII and OpenAI built general models; Shizuku AI built one for a single character, who already has an audience on YouTube, X and Discord.

For anyone choosing what to build on, the trade-off is plain. The two academic models are far weaker on the company’s content rows, but their weights are public. Shizuku-Duplex looks stronger on the company’s own numbers, and for now nobody outside the company can use it.

6. What is public so far

As of November 10, 2026, Shizuku-Duplex is an announcement with recordings and a self-run benchmark. The post links no weights, code, paper or API, and we found no Shizuku-Duplex repository on Hugging Face or GitHub. Nor does it say how or when the model will reach Shizuku’s streams or apps, beyond “Using this technology, we are developing Shizuku.” The next steps it names are better dialogue, vision input and support for other languages.

The most useful next step for everyone else would be a public demo, an API, weights or a paper, any of which would let others check the timing and content results. Releasing the Japanese test set would also help. Public Japanese full-duplex benchmarks are scarce, as the post itself says, and this one would let other labs rerun the six-model comparison with their own models added.

The company’s own plans point to other languages. a16z described multilingual voice synthesis as part of the plan in February, and the post lists other languages as a next step, without saying whether they will come as versions of Shizuku-Duplex.

Sources

• Shizuku AI, “Shizuku-Duplex: Japanese Full-Duplex Model for Natural Conversation”, November 8, 2026, and its Japanese edition.
• Shizuku AI’s YouTube channel.
• Shizuku AI on X, announcement post, November 8, 2026.
• Shizuku AI, careers page, last updated July 13, 2026.
• Justine Moore, Marco Mascorro and Martin Casado, “Investing in Shizuku AI”, a16z, February 9, 2026.
• The Bridge, February 12, 2026, on the seed round.
• ASCII.jp, November 9, 2026, on the iPhone prototype and Shizuku-Duplex.
• A. Kodaira et al., StreamDiffusion, arXiv 2312.12491, and the ICCV 2025 paper.
• OpenAI, “Introducing GPT-Live”, July 8, 2026.
• Thinking Machines Lab, “Interaction Models: A Scalable Approach to Human-AI Collaboration”, May 11, 2026.
• A. Défossez et al., Moshi, arXiv 2410.00037, and the Moshi README.
• S. Kuroki et al., KAME, ICASSP 2026.
• NII, LLM-jp-Moshi-v1 press release (Japanese), February 25, 2026, and the Hugging Face cards for LLM-jp-Moshi-v1 and J-Moshi.

Next: Dessa at Block: a payments company's research lab says it trained a full-duplex model on speech alone