Is this page helpful?

Cloning a Voice with Zero-Shot TTS#

The Magpie TTS Zeroshot model synthesizes speech that matches the voice characteristics of a short reference audio recording, with multilingual voice cloning in 12 languages. This guide covers how to prepare an audio prompt, tune quality settings, and synthesize cloned speech through gRPC, HTTP, and WebSocket.

Prerequisites#

A deployed TTS NIM running the Magpie TTS Zeroshot model. Refer to the TTS tutorial for deployment steps.
Installed the NVIDIA Riva Python client.

Note

Magpie TTS Zeroshot requires access approval. Request access through this form.

Model Capabilities#

Capability	Magpie TTS Zeroshot
Languages	12 languages
Inference modes	Streaming + Offline
Audio prompt	Required
Quality parameter	1–40 (default: 20)
Built-in voices	2 (Female and Male)
GPU memory	13.06 GB (`batch_size=8`)

Magpie TTS Zeroshot supports English (en-US), Spanish (es-US), French (fr-FR), German (de-DE), Mandarin (zh-CN), Vietnamese (vi-VN), Italian (it-IT), Hindi (hi-IN), Japanese (ja-JP), Arabic (ar-XA), Portuguese (pt-BR), and Korean (ko-KR).

Prepare the Audio Prompt#

The audio prompt is the reference recording whose voice characteristics the model will replicate.

Requirements:

Format: 16-bit mono WAV
Sample rate: 22.05 kHz or higher
Duration: 3–10 seconds (aim for approximately 5 seconds)
Content: Clear speech with minimal background noise

Tips for higher quality:

Trim silence from the beginning and end so speech fills most of the prompt.
Record in a quiet, echo-free environment.
Use consistent volume throughout the recording.
Avoid music, sound effects, or overlapping speakers.

Synthesize with Magpie TTS Zeroshot#

Offline (gRPC)#

python3 python-clients/scripts/tts/talk.py --server 0.0.0.0:50051 \
    --language-code en-US \
    --text "This speech uses a cloned voice from my audio prompt." \
    --voice Magpie-ZeroShot-Multilingual \
    --zero_shot_audio_prompt_file prompt.wav \
    --output output.wav

Offline (HTTP)#

curl -sS http://localhost:9000/v1/audio/synthesize --fail-with-body \
    -F language=en-US \
    -F text="This speech uses a cloned voice from my audio prompt." \
    -F voice=Magpie-ZeroShot-Multilingual \
    -F audio_prompt=@prompt.wav \
    --output output.wav

Note

The @ prefix on the file path is required by curl for file uploads.

Streaming (gRPC)#

python3 python-clients/scripts/tts/talk.py --server 0.0.0.0:50051 \
    --language-code en-US \
    --text "This speech uses a cloned voice from my audio prompt." \
    --voice Magpie-ZeroShot-Multilingual \
    --zero_shot_audio_prompt_file prompt.wav \
    --stream \
    --output output.wav

Streaming (WebSocket)#

The WebSocket client uses hyphens instead of underscores for argument names.

python3 python-clients/scripts/tts/realtime_tts_client.py \
    --server localhost:9000 \
    --language-code en-US \
    --text "This speech uses a cloned voice from my audio prompt." \
    --voice Magpie-ZeroShot-Multilingual \
    --zero-shot-audio-prompt-file prompt.wav \
    --output output.wav

Tune Quality Settings#

The Magpie TTS Zeroshot model accepts a --zero_shot_quality parameter (range: 1–40, default: 20) that controls the trade-off between synthesis speed and voice similarity.

Lower values (1–10): Faster synthesis, lower voice fidelity.
Default (20): Balanced quality and speed.
Higher values (21–40): Slower synthesis, closer voice match.

python3 python-clients/scripts/tts/talk.py --server 0.0.0.0:50051 \
    --language-code en-US \
    --text "Higher quality voice cloning." \
    --voice Magpie-ZeroShot-Multilingual \
    --zero_shot_audio_prompt_file prompt.wav \
    --zero_shot_quality 30 \
    --output output.wav

For the WebSocket client, the equivalent flag is --zero-shot-prompt-quality.

Use Built-In Voices Instead#

Magpie TTS Zeroshot also includes built-in voices that do not require an audio prompt. Specify a voice name with the --voice flag instead of an audio prompt file.

python3 python-clients/scripts/tts/talk.py --server 0.0.0.0:50051 \
    --language-code en-US \
    --text "Using a built-in voice." \
    --voice Magpie-ZeroShot-Multilingual.Female \
    --output output.wav

Available voices: Magpie-ZeroShot-Multilingual.Female (default) and Magpie-ZeroShot-Multilingual.Male. The same built-in voices are available for every supported language.

For the complete voice list, refer to the TTS support matrix.

Key Differences Between Scripts#

The two Python client scripts use different argument naming conventions:

Parameter	`talk.py` (gRPC)	`realtime_tts_client.py` (WebSocket)
Audio prompt	`--zero_shot_audio_prompt_file`	`--zero-shot-audio-prompt-file`
Transcript	`--zero_shot_transcript`	`--zero-shot-audio-prompt-transcript`
Quality	`--zero_shot_quality`	`--zero-shot-prompt-quality`
Server	`--server 0.0.0.0:50051`	`--server localhost:9000`

Limitations#

The audio prompt transcript parameters (--zero_shot_transcript and --zero-shot-audio-prompt-transcript) are still accepted by the clients and the API, but no currently supported TTS model uses the value. Magpie TTS Zeroshot ignores it.
Audio prompts shorter than 3 seconds or longer than 10 seconds can produce lower quality results.