NVIDIA releases NemotronLabs VoiceChat 11B, an open full-duplex voice model with tool calling

2026-08-03★★★after cutoffopen-sourceNVIDIAconfidence: medium

NVIDIA published NVIDIA-NemotronLabs-VoiceChat-11B on Hugging Face (card dated 2026-08-03; arXiv 2609.21967): an end-to-end, full-duplex speech-to-speech model (FastConformer encoder + Nemotron Nano v2 9B + TTS decoder) that NVIDIA calls the first open full-duplex model to support tool calling. It has ~450 ms turn-taking latency and ranks #2 among open models on VoiceBench and Full-Duplex-Bench.

Key facts

11B params; English; OpenMDW-1.1 license
Tool calling: BFCL-v3 (AU Harness) 56.1%; Full-Duplex-Bench v3 tool selection 82.5%
Turn-taking ~450 ms; interruption latency 480 ms; smooth turn-taking 0.82 (FDB 1.0)
Part of NVIDIA's 2026 Nemotron Speech push: PersonaPlex-7B (Jan, Moshi-based), Nemotron Speech Streaming ASR, Nemotron 3.5 ASR (40 locales, June)

What happened

NVIDIA added an 11B end-to-end full-duplex voice model to its Nemotron Speech collection. It listens and speaks at the same time, and it can call external tools while keeping the conversation going. Before this, open full-duplex models did not do tool calling.

Why it matters

Open full-duplex models (Kyutai Moshi, NVIDIA PersonaPlex) were mostly chat demos. Tool calling makes an open, self-hostable alternative to cascaded ASR→LLM→TTS agents and to closed realtime APIs possible. The "first" is NVIDIA's own claim. The release date comes from the model card; we found no separate press release.

Changelog

2026-09-29: created

Models

NVIDIA NemotronLabs VoiceChat 11B (and PersonaPlex-7B) NVIDIA · current

Related events

2026-09-15Google ships Gemini 3.8 Live voice models and Gemini 3.8 Flash TTS with voice design and cloning ★★
2026-07-08OpenAI launches GPT-Live, full-duplex voice models replacing ChatGPT's Advanced Voice Mode ★★★★

Sources (3)

id: 2026-08-03-nvidia-nemotronlabs-voicechat · updated 2026-09-29 · open in the interactive timeline