> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/labs-voice-agent/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/labs-voice-agent/_mcp/server. # Release Notes for NeMo Voice Agent Dated feature updates for NeMo Labs Voice Agent appear newest first. This page is the documentation site's canonical history of automatic speech recognition (ASR), large language model (LLM), text-to-speech (TTS), and evaluation milestones. There are no per-entry version tags: the package version is a single value declared in `nemo_voice_agent/package_info.py` and exposed as `nemo_voice_agent.package_info.__version__`. For changes after the most recent entry, refer to the repository commit history. ## Release History The following table summarizes the dated project milestones documented on this page. | Date | Highlights | | --- | --- | | 2026-08-06 | Graduated to a standalone repository | | 2026-06-13 | Voice-agent evaluation harness with four benchmark domains | | 2026-05-15 | Omni (multimodal) LLM support | | 2026-01-26 | Nemotron-3-Nano-30B-A3B-BF16 LLM and Magpie multilingual TTS | | 2025-12-31 | Tool-calling examples and diarization model upgrade | | 2025-11-14 | Joint ASR and end-of-utterance detection | | 2025-10-10 | Kokoro-82M TTS | | 2025-10-03 | vLLM serving with automatic Hugging Face fallback | | 2025-09-05 | First release | ## 2026-08-06 — Standalone Repository NeMo Labs Voice Agent graduated out of [NVIDIA-NeMo/Speech](https://github.com/NVIDIA-NeMo/Speech/tree/main/examples/voice_agent), where it had lived as a sub-package. Development now occurs in its own repository. The Python package name (`nemo_voice_agent`) and every import path are unchanged. ## 2026-06-13 — Evaluation Harness This milestone shipped the two-bot evaluation harness. A simulated user bot talks to the agent under test through a bridge that transfers audio between two WebSocket Pipecat servers and scores the outcome. The release ported four benchmark domains: | Domain | Scenarios | Upstream Source | | --- | --- | --- | | `eva_airline` | 50 | ServiceNow `eva` | | `tau2_airline` | 50 | tau2-bench airline | | `tau2_retail` | 114 | tau2-bench retail | | `tau2_telecom` | 114 | tau2-bench telecom (dual-side: separate agent-side and user-side databases) | `tau2_telecom` is registered twice — a second `tau2_telecom_workflow` family shares the same 114 upstream tasks and differs only in which policy file is rendered into the agent prompt. Scoring is per-scenario: each scenario declares a `success_signals` whitelist over the six signals `is_action_match`, `db_state_match`, `db_state_assertion`, `nl_assertion`, `judge_passed`, and `clean_exit`. Learn more: [Evaluation overview](/nemo/labs-voice-agent/evaluate-voice-agents/overview) · [Benchmarks](/nemo/labs-voice-agent/evaluate-voice-agents/understand-scoring/benchmarks-domains) · [Scoring](/nemo/labs-voice-agent/evaluate-voice-agents/understand-scoring/scoring-model) · [eva_airline](/nemo/labs-voice-agent/evaluate-voice-agents/domain-guides/eva-airline) · [tau2_airline](/nemo/labs-voice-agent/evaluate-voice-agents/domain-guides/tau-2-airline) · [tau2_retail](/nemo/labs-voice-agent/evaluate-voice-agents/domain-guides/tau-2-retail) · [tau2_telecom](/nemo/labs-voice-agent/evaluate-voice-agents/domain-guides/tau-2-telecom) ## 2026-05-15 — Omni and Multimodal LLM Support Added support for `nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4`, served through the `llm_configs/nemotron_nano_v3_omni.yaml` model config (a `_think.yaml` sibling enables reasoning). Omni models consume user audio directly, so the pipeline inserts a user audio buffer stage that non-omni pipelines omit. Learn more: [Multimodal models](/nemo/labs-voice-agent/about/core-concepts/language-models/multimodal-omni) · [Reasoning](/nemo/labs-voice-agent/about/core-concepts/language-models/reasoning-mode) ## 2026-01-26 — Nemotron-3-Nano-30B-A3B and Magpie TTS This milestone added the following language and speech model support. - Added support for the `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16` LLM. The `llm_configs/nemotron_nano_v3.yaml` model configuration covers this family. The NVFP4 variant is the shipped default and needs a GPU with FP4 support. - Added support for the `nvidia/magpie_tts_multilingual_357m` TTS model. Its configuration is `tts_configs/magpie_tts_multilingual_357m.yaml`. Learn more: [LLM backends](/nemo/labs-voice-agent/about/core-concepts/language-models/llm-backends) · [TTS](/nemo/labs-voice-agent/about/core-concepts/speech-pipeline/text-to-speech) · [Model registry](/nemo/labs-voice-agent/build-voice-agents/configure/model-registry) ## 2025-12-31 — Tool-Calling Examples and Diarization Upgrade This milestone expanded tool-calling examples and updated the default diarization model. - Added tool-calling examples. The Kokoro TTS service lets the LLM change how the agent sounds mid-conversation. It registers `tool_tts_speak_faster`, `tool_tts_speak_slower`, `tool_tts_set_speed`, `tool_tts_reset_speed`, `tool_tts_set_voice` (American or British accent, male or female voice), and `tool_tts_reset_voice`. A standalone example function, `tool_get_city_weather`, looks up current weather with the `python_weather` package. - Updated the diarization model to `nvidia/diar_streaming_sortformer_4spk-v2.1`, which handles up to four speakers across user turns. Learn more: [Tool calling](/nemo/labs-voice-agent/build-voice-agents/tools/tool-calling) · [Custom tools](/nemo/labs-voice-agent/build-voice-agents/tools/writing-your-own-tools) · [Diarization](/nemo/labs-voice-agent/about/core-concepts/speech-pipeline/speaker-diarization) ## 2025-11-14 — Joint ASR and End-of-Utterance Detection Added support for `nvidia/parakeet_realtime_eou_120m-v1`, which predicts end-of-utterance jointly with transcription. It is the shipped default STT model and is optimized for lowest latency. It does not emit punctuation or capitalization. Learn more: [ASR](/nemo/labs-voice-agent/about/core-concepts/speech-pipeline/speech-recognition) · [Turn taking](/nemo/labs-voice-agent/about/core-concepts/speech-pipeline/turn-taking-backchannels) ## 2025-10-10 — Kokoro-82M TTS Added support for the `hexgrad/Kokoro-82M` TTS model, configured through `tts_configs/kokoro_82M.yaml`. Learn more: [TTS](/nemo/labs-voice-agent/about/core-concepts/speech-pipeline/text-to-speech) ## 2025-10-03 — vLLM Serving and Automatic Backend Selection This milestone added the following serving and backend-selection capabilities. - Added vLLM as an LLM serving backend alongside Hugging Face. - Added automatic backend selection: with `llm.type: auto`, the server tries vLLM first and falls back to Hugging Face, so parameters for both backends must be valid. - Made `nvidia/NVIDIA-Nemotron-Nano-9B-v2` the default LLM at the time. The current default is the Nemotron-3-Nano-30B-A3B family, as described in the 2026-01-26 entry. Learn more: [vLLM backend](/nemo/labs-voice-agent/build-voice-agents/model-serving/serving-with-v-llm) · [LLM backends](/nemo/labs-voice-agent/about/core-concepts/language-models/llm-backends) ## 2025-09-05 — First Release First release of NeMo Labs Voice Agent: a real-time speech-to-speech pipeline built on Pipecat, with streaming ASR, speaker diarization, LLM, and TTS behind a WebSocket server, plus a browser client. Shipped at the time as a sub-package of [NVIDIA-NeMo/Speech](https://github.com/NVIDIA-NeMo/Speech/tree/main/examples/voice_agent). Refer to the 2026-08-06 entry for the move to a standalone repository. Learn more: [Architecture](/nemo/labs-voice-agent/about/architecture) · [Quickstart](/nemo/labs-voice-agent/get-started/quickstart) ## Upcoming Work The repository `README.md` lists the following planned work: - Accuracy and robustness improvements to the ASR models. - Combined ASR and speaker diarization to handle overlapping speech. - More evaluation domains and scenarios. ## Related Topics Use these pages to begin with the current project, understand its architecture, or review its evaluation capabilities. - [Quickstart](/nemo/labs-voice-agent/get-started/quickstart) — run the current default configuration. - [Architecture overview](/nemo/labs-voice-agent/about/architecture) — understand the runtime pipeline and services. - [Evaluation overview](/nemo/labs-voice-agent/evaluate-voice-agents/overview) — understand the evaluation workflow and scoring signals.