Overview | NeMo Labs Voice Agent

The speech pipeline turns live microphone audio into text, decides when the user has finished speaking, and turns the assistant response back into audio. The stages stream data rather than waiting for an entire turn. They include voice activity detection (VAD), automatic speech recognition (ASR), speaker diarization, turn taking, large language model (LLM) inference, and text-to-speech (TTS).

WebSocket audio -> VAD -> ASR -> diarization -> turn taking -> LLM -> TTS -> WebSocket audio

Optional stages are omitted by the builders when their configuration disables them. For the complete runtime ordering and the frames exchanged between stages, refer to Architecture.

Core Concepts

The following pages explain the stages that interpret incoming audio and produce spoken output.

Concept	What It Explains
Speech recognition	Streaming transcription, model selection, and end-of-utterance tokens.
Speaker diarization	Assigning speaker identities across user turns.
Text to speech	Streaming synthesis, voices, and text aggregation.
Turn taking	VAD, end-of-utterance signals, interruptions, and backchannels.

Use these pages to experience the pipeline, change its configuration, or understand the complete architecture.

Quickstart — run the complete pipeline.
Configure — select models and tune pipeline behavior.