Is this page helpful?

Chatterbox TTS Plugin Samples#

The NVIGI Chatterbox TTS Plugin pack includes a 3D-rendered GUI-based sample application that demonstrates how to integrate Text-to-Speech (TTS) into your applications, along with optional ASR (Automatic Speech Recognition) and GPT (Large Language Model) integration for voice-to-voice AI interactions.

Overview#

The sample demonstrates the use of the NVIGI Chatterbox TTS plugin with CUDA, Vulkan, and D3D12 backends. The sample also integrates GPT and Whisper ASR plugins to create a complete voice-to-voice AI pipeline. The sample is provided in source form within the package, allowing you to build and customize it for your needs.

The 3D Sample#

The 3D sample, nvigi.3d, combines NVIGI and Donut (https://github.com/NVIDIAGameWorks/donut) to create a sample app demonstrating NVIGI TTS integration in a 3D application. Using NVIGI, it’s possible to support multiple backends within a single application. The sample shows this with CUDA, Vulkan, and D3D12 TTS backends.

The Donut-based NVIDIA NVIGI (In-Game Inference) 3D Sample is an interactive 3D application designed to show how one might integrate text-to-speech features into a UI-based workflow. The focus is on presenting options to the user and running TTS workflows without blocking the 3D interaction or rendering. The sample defaults to rendering with Direct3D 12 but can switch to Vulkan via a command-line option. Both rendering modes support the full ASR + GPT + TTS voice-to-voice pipeline.

Requirements#

  • Hardware:

    • GPU: NVIDIA GPU with Compute Capability 7.0+ (RTX 20x0 series or newer)

    • Memory: 16 GB RAM minimum

    • VRAM: ~2.3 GB for Turbo TTS-only, ~3.5 GB for Multilingual TTS-only, ~8.5 GB+ for full pipeline (ASR + GPT + TTS)

  • Software:

    • Windows 10/11 (64-bit)

    • NVIDIA graphics driver r580 or newer

    • Visual Studio 2022 with C++ development tools (for rebuilding samples)

Setting up and Launching the Sample#

The sample requires the Chatterbox TTS model (and optionally GPT and ASR models) to be available in the models directory. The models are included in the data/nvigi.models directory.

Launching the Pre-Built Sample#

For those using the prebuilt binaries in the package, the sample executable is available immediately and can be run:

  1. Navigate to <PACK_ROOT>/bin/x64/Release/

  2. Run .\nvigi.3d.exe, either by double-clicking the executable in Windows Explorer or by running it from a command prompt

The sample requires and looks for AI models and rendering media relative to the executable path. Specifically, it looks for:

  • The models directory: <PACK_ROOT>/data/nvigi.models

  • The media directory: <PACK_ROOT>/data/nvigi.test/nvigi.3d

  • Speaker embeddings: <PACK_ROOT>/data/nvigi.test/spk_emb

The code will check upward from the executable directory several times to find these directories. This is done so that the pre-built binary layout is trivially supported with no user effort.

If required, the paths may be specified explicitly via command line arguments:

  • -pathToModels <path> - Path to the models directory

  • -pathToTestData <path> - Path to test data including speaker embeddings

Using the Sample#

Main UI#

On launch, the sample will show a UI box on the left side of the window and a 3D rendered scene. The main UI contains:

  • Performance Info: GPU, system and performance statistics at the top

  • Model Status: Current ASR, GPT, and TTS model/backend status displayed directly

  • Options Button: Opens a popup with tabs for App settings, ASR, GPT, and TTS configuration

  • Chat Area: Scrollable message history showing questions (Q:) and answers (A:)

  • Text Input: Single-line text input field (press Enter to submit)

  • Voice Controls: Record/Stop button for voice input (when ASR is available)

  • Reset Chat: Button to clear conversation history (when GPT is available)

  • Performance Metrics: Real-time display of ASR, GPT, and TTS timing statistics

Operating Modes#

The 3D sample supports two graphics backends, both providing the full voice-to-voice AI pipeline:

D3D12 Mode (Default)#

When running with D3D12 graphics (default), the sample uses D3D12 backends for inference when available, with CUDA as fallback:

ASR → GPT → TTS Pipeline:

  1. Voice Input: Click “Record” to capture audio from your microphone

  2. ASR Processing: Whisper ASR transcribes your speech to text

  3. GPT Processing: The transcribed text is sent to the GPT model for a response

  4. TTS Output: Chatterbox TTS converts the GPT response to speech

Text Input:

  • Enter text in the input field and press Enter to submit

  • When GPT is available: Text is sent to GPT for a response, which is then spoken via TTS

  • The conversation history is displayed in the chat area with questions (Q:) and answers (A:)

Backend Selection:

  • D3D12 backends are preferred for TTS, GPT, and ASR when available (requires driver 580.61 or newer)

  • CUDA backends are used as fallback if D3D12 plugins are unavailable or fail compatibility checks

  • D3D12 backends provide shared context with graphics for optimal GPU scheduling

  • This mode supports CUDA in Graphics (CIG) optimization for efficient GPU sharing between rendering and inference

Note: D3D12 backend performance characteristics:

  • First inference may take approximately 6 seconds (one-time shader compilation cost).

  • Changing speaker embeddings may add approximately 1 second delay to the first inference after the change.

Mitigation: Run a one-off dummy inference during your application’s loading screen — and once per speaker embedding you intend to use — so the user-visible inference always runs at steady-state speed. See D3D12 Warmup Latency for the full recipe and additional strategies.

Vulkan Mode#

When running with -vk flag, the sample uses Vulkan backends for inference:

.\nvigi.3d.exe -vk

Full Pipeline Support:

  • All features of D3D12 mode are available: ASR, GPT, and TTS

  • Voice input and text input work the same as D3D12 mode

  • TTS, GPT, and ASR use Vulkan backends for inference

This mode is useful for testing the Vulkan backend or for broader GPU compatibility.

Note: When attempting to share the Vulkan device between the graphics pipeline and the Chatterbox TTS plugin, the generated output may be incorrect. The plugin creates its own separate Vulkan device (without sharing) to ensure correct output.

Voice Input#

The voice input feature allows you to speak to the AI and hear responses (available in both D3D12 and Vulkan modes when ASR is configured):

  1. Click the “Record” button to start recording (button changes to “Stop”)

  2. Speak your question or statement

  3. Click “Stop” when finished

  4. The ASR model transcribes your speech

  5. If GPT is available, the transcription is sent to GPT for a response, which is then spoken via TTS

  6. If only TTS is available (no GPT), the transcribed text is spoken directly via TTS

Note: Voice input requires a working microphone. The sample uses the system default microphone.

Text Input#

For text-based interaction:

  1. Enter your text in the single-line text input field

  2. Press Enter to submit:

    • With GPT available: Text is sent to GPT, and the response is spoken via TTS

    • Without GPT: Text is converted directly to speech via TTS

  3. Click Reset Chat to clear the conversation history (when GPT is available)

Speaker Selection#

The sample supports multiple speaker voice profiles via speaker embeddings:

  1. Click the Options… button to open the settings popup

  2. Select the TTS tab

  3. Use the Voice Name dropdown to select from available voices

  4. Speaker embeddings are JSON files located in data/nvigi.test/spk_emb/

  5. The selected voice will be used for all subsequent TTS output

To add custom voices, place speaker embedding JSON files in the spk_emb directory.

Options Popup#

Click the Options… button to open the settings popup. The popup contains tabs for different settings categories:

App Tab#

Application-level settings:

  • GPU Scheduling Priority: A dropdown to set the 3D-vs-compute/inference prioritization:

    • Prioritize Graphics: Give more GPU priority to 3D rendering, at the expense of inference latency. Use this when maintaining a smooth frame rate is critical and higher TTS latency can be tolerated (e.g., visually intensive scenes with background narration).

    • Prioritize Inference: Give more GPU priority to compute, improving inference latency at the potential expense of rendering time. Use this when low TTS latency is critical and some frame drops can be tolerated (e.g., real-time conversational AI where responsiveness matters most).

    • Balanced: A more even split between the two. This is a good default for most applications, providing reasonable frame rates and TTS latency without heavily favoring either workload.

  • Frame Rate Limiter: If checked, allows you to specify the target FPS to avoid the sample running at excessively high frame rates. The 3D sample has minimal graphics load, so without limiting, it may run at hundreds of FPS. This doesn’t reflect real-world game scenarios where the GPU is under heavier load. By limiting the frame rate (e.g., to 60 or 120 FPS), you can simulate more realistic conditions and better evaluate how TTS performance and GPU scheduling modes will behave in an actual game with demanding graphics.

ASR Tab#

Automatic Speech Recognition settings:

  • Automatic Backend Selection: When enabled, automatically selects the best available backend

  • VRAM Budget: Set the VRAM budget (in MB) for ASR model loading

  • Model Name: Select from available Whisper ASR models

GPT Tab#

GPT (Large Language Model) settings:

  • Automatic Backend Selection: When enabled, automatically selects the best available backend

  • VRAM Budget: Set the VRAM budget (in MB) for GPT model loading

  • Model Name: Select from available GPT models

TTS Tab#

Text-to-Speech settings:

  • Automatic Backend Selection: When enabled, automatically selects the best available backend

  • VRAM Budget: Set the VRAM budget (in MB) for TTS model loading

  • Model Name: Select from available Chatterbox TTS models (Turbo or Multilingual)

  • Voice Name: Select from available speaker embedding profiles

  • Language: When a multilingual model is selected, a language dropdown appears with 23 supported languages

When switching between Turbo and Multilingual models, the voice selection automatically adjusts to a compatible speaker embedding.

Note that when inference is actively running, the settings will be grayed out and disabled for interaction.

Performance Logging#

The sample includes comprehensive performance logging that tracks:

  • ASR transcription time and transcribed text

  • GPT first token latency and generated response

  • TTS total inference time, first audio latency, and audio duration generated

Performance metrics are displayed in real-time in the UI and also written to a CSV log file (nvigi_perf_log_YYYYMMDD_HHMMSS.csv) in the executable directory for analysis. The CSV format includes columns for Timestamp, InferenceType, TimeMs, and AdditionalInfo.

Logging from the Sample#

By default, the pre-built sample will launch a log window that shows NVIGI log messages during init, creation, runtime and shutdown. In addition, logging to file may be enabled by specifying a path:

.\nvigi.3d.exe -logToFile ..\logs

With this option, a log file would be written to ..\logs\nvigi-log.txt

Logs are useful when debugging issues or sending support questions.

Building the Sample from Source#

While the pre-built 3D sample is ready to run out of the box, building from source offers several advantages:

  • Debugging: Step through the code in Visual Studio to understand the TTS integration flow, inspect variables, and diagnose issues

  • Customization: Modify the sample to experiment with different TTS parameters, UI layouts, or integration patterns for your own application

  • Learning: Study and modify the code to better understand NVIGI plugin loading, TTS callbacks, text chunking, and real-time audio playback handling

  • Testing: Validate fixes or test new features before integrating them into your own project

The source code for the 3D Sample is provided in <PACK_ROOT>/source/samples/nvigi.3d. To build the sample:

Prerequisites#

  1. Ensure Visual Studio 2022 is installed with C++ development tools

Build Steps#

  1. Open a command prompt or VS2022 Developer Command Prompt

  2. Navigate to <PACK_ROOT>

  3. Run .\setup_sample.bat to download the NVIGI Core SDK and set up dependencies. The NVIGI Developer Full SDK path is required when building the 3D sample (or the full solution) because the 3D project includes headers (nvigi_gpt.h, nvigi_asr_whisper.h) and plugin DLLs that are only available through that SDK. It is optional for a TTS-only build (plugins + CLI sample).

    :: TTS-only build (plugins + nvigi.tts.chatterbox.exe)
.\setup_sample.bat

:: Full build including the 3D sample — Developer Full SDK path is REQUIRED
.\setup_sample.bat "C:\path\to\nvigi_developer_full_sdk"

    When the SDK path is provided, setup_sample.bat also stages the GPT/ASR plugin DLLs and creates a junction at external\nvigi_developer_full_sdk_win64 that the 3D project’s premake.lua resolves header paths against. Without the path, build.bat -Release will fail the 3D target with cannot open include file: 'nvigi_asr_whisper.h' (and 'nvigi_gpt.h'); rerun setup_sample.bat with the SDK path and re-run setup.bat.

  4. Run .\setup.bat to generate the Visual Studio solution

  5. Run .\build.bat -Release (or -Production) to build all projects, or:

    • Open <PACK_ROOT>/_project/vs2022/nvigi.chatterbox.sln in Visual Studio 2022

    • Select your desired configuration (Debug, Release, Production)

    • Build the nvigi.3d project

  6. Run .\copy_sdk_binaries.bat Release (or Production) to copy the built executables and DLLs to <PACK_ROOT>/build_bin/x64/Release

The built sample executable will be available at <PACK_ROOT>/build_bin/x64/Release/nvigi.3d.exe.

Building with Local Donut#

The default build uses a pre-built version of the Donut rendering framework. It is also possible to rebuild the Sample against a locally-built version of Donut. To do this:

  1. Open a VS2022 Developer Command Prompt and navigate to <PACK_ROOT>/source/samples/nvigi.3d/opt-local-donut

  2. Download the Donut source code by running .\01_pull.bat

  3. Run .\02_setup.bat to run CMake to create the build files

  4. Run .\03_build.bat to build all three configurations of Donut

  5. Edit <PACK_ROOT>/source/samples/nvigi.3d/premake.lua to:

    1. Comment out donut_dir = externaldir.."donut"

    2. Uncomment donut_dir = ROOT.."source/samples/nvigi.3d/opt-local-donut/_package/donut"

  6. Re-run <PACK_ROOT>\setup.bat

  7. Rebuild the solution

  8. Rerun .\copy_sdk_binaries.bat Release (or Production) to copy the binaries

Running the Built Sample in Debugger#

To run the rebuilt sample from within the Visual Studio debugger:

  1. One-time setup in the project file (needs to be redone if _project is deleted):

    1. In the Visual Studio IDE, edit the project config settings for nvigi.3d

    2. Navigate to the “Debugging” settings

    3. Set “Command” to $(SolutionDir)..\..\bin\x64\Release\nvigi.3d.exe

    4. Set “Command Arguments” as needed (see command-line options below)

    5. Set “Working Directory” to $(SolutionDir)..\..\bin\x64\Release

  2. Build the desired configuration (Release is recommended - it is optimized but contains symbols)

  3. After each (re-)build, re-run .\copy_sdk_binaries.bat Release to copy the updated binaries

  4. Set nvigi.3d as the startup project in Visual Studio

  5. Launch with debugging (F5)

Command-line Options#

Arguments

Effect

-pathToModels <path>

Path to the models directory. Defaults to searching upward from executable directory for data/nvigi.models

-pathToTestData <path>

Path to test data directory containing speaker embeddings and media

-logToFile <directory>

Sets the destination directory for logging

Additional Command Line Arguments#

Arguments

Effect

-vk

Use Vulkan for rendering (full ASR/GPT/TTS pipeline with Vulkan backends)

-width 1920

Sets window width

-height 1080

Sets window height

-verbose

Allows verbose info level logging

-debug

Enables NVRHI and Graphics API validation Layer

-noSigCheck

Does not do NVIGI dll signature check

-vsync

Enables Vsync

-scene "/myscene.fbx"

Loads a custom scene

-maxFrames 100

Sets number of frames to render before the app shuts down

-noCIG

Disable the use of CUDA in Graphics optimization (for debugging/testing purposes)

Input Text Guidelines#

For best TTS output quality, provide grammatically complete sentences with standard punctuation. The following input patterns may produce reduced quality or garbled audio:

  • Incomplete words or fragments (e.g., "the", "that", "hundred") — the model may not treat these as complete utterances and can generate extra, unintelligible audio.

  • Very short utterances of fewer than ~5 words such as "Yes", "Help!", or "Over here!" — common in game dialogue but more likely to produce artifacts because the model has too little context to anchor prosody.

  • Repeated or excessive punctuation (e.g., Hello!!!, Really?!) — multiple punctuation marks in sequence can confuse the tokenizer.

Use complete sentences of moderate to long length (>5 words) with single punctuation marks for the most reliable results. For short game dialogue specifically, see Mitigations for short game dialogue in the Programming Guide for concrete strategies (text padding, paralinguistic anchors, pre-rendered barks).

Notes#

  • Both D3D12 and Vulkan modes support the full ASR/GPT/TTS voice-to-voice pipeline

  • D3D12 mode: Uses D3D12 backends for TTS, GPT, and ASR when available (requires driver 580.61+), with CUDA as fallback

  • Vulkan mode: Uses Vulkan backends for TTS, GPT, and ASR

  • The sample is designed for use with local systems - use on remote desktop is not recommended

  • First inference may take longer due to model warmup

  • D3D12 backends provide shared context with graphics for optimal GPU scheduling

  • D3D12 backend delays: First inference may take approximately 6 seconds (one-time shader compilation), and changing speaker embeddings adds approximately 1 second to the first inference after the switch. Mitigation: pre-warm with a dummy inference during the loading screen and pre-warm each speaker embedding the level uses; see D3D12 Warmup Latency for the recommended pattern.

  • Vulkan device sharing limitation: When attempting to share the Vulkan device between the graphics pipeline and the Chatterbox TTS plugin, the generated output may be incorrect. The plugin creates its own separate Vulkan device (without sharing) to ensure correct output.

The CLI Sample#

The CLI sample, nvigi.tts.chatterbox, demonstrates TTS usage with both Chatterbox Turbo and Multilingual models and writes a 24 kHz mono WAV file. It is intended as a minimal, scriptable example for batch generation or quick validation. The sample also supports an interactive mode with real-time audio playback on Windows.

Requirements#

  • The Chatterbox TTS model(s) in data/nvigi.models

  • A speaker embedding JSON file in data/nvigi.test/spk_emb

Launching the Pre-Built Sample#

The pre-built CLI sample is available in the package. The sample automatically detects the NVIGI SDK DLLs from the executable directory.

Turbo (English-only, default):

cd <PACK_ROOT>\bin\x64\Release
nvigi.tts.chatterbox.exe --models ..\..\..\data\nvigi.models --speaker ..\..\..\data\nvigi.test\spk_emb\aaron_turbo.json --text "Hello from Chatterbox Turbo." --output output.wav

Multilingual (23 languages):

nvigi.tts.chatterbox.exe --models ..\..\..\data\nvigi.models --speaker ..\..\..\data\nvigi.test\spk_emb\french.json --multilingual --language fr --text "Bonjour, je suis un modèle multilingue." --output output_fr.wav

The --multilingual flag switches to the multilingual model, and --language specifies the language code (e.g., en, fr, de, zh, ja, hi). The model GUID is automatically selected based on the --multilingual flag.

The sample outputs timing statistics after generation, for example (measured on an NVIDIA GeForce RTX 5080 with the CUDA backend and the Turbo model speaking “Hello from Chatterbox Turbo.”):

Inference Time: 393.40 ms
Audio Length:   1820.00 ms (1.82 sec)
RTF:            0.22
Wrote WAV: <absolute path to the file passed via --output>

The RTF (Real-Time Factor) indicates performance: values below 1.0 mean the audio was generated faster than real-time. Absolute numbers will vary with hardware, driver, backend, and text length.

Interactive Mode (Windows Only)#

The CLI sample supports an interactive mode that allows continuous text-to-speech generation with real-time audio playback:

Turbo:

nvigi.tts.chatterbox.exe --models ..\..\..\data\nvigi.models --speaker ..\..\..\data\nvigi.test\spk_emb\aaron_turbo.json --interactive --backend cuda

Multilingual:

nvigi.tts.chatterbox.exe --models ..\..\..\data\nvigi.models --speaker ..\..\..\data\nvigi.test\spk_emb\german.json --interactive --multilingual --language de

In interactive mode:

  • Enter text prompts at the > prompt to synthesize speech

  • Audio plays immediately through your default audio device

  • Timing statistics are displayed after each generation

  • For multilingual mode, enter text in the selected language (UTF-8 console input is enabled automatically)

  • Type exit, quit, or press Enter on an empty line to exit

Batch mode (JSONL)#

For large fixed sets of prompts (e.g. WER/regression), use --batch <manifest.jsonl> with --multilingual and a backend (--backend / -b). Each line of the file is one JSON object with:

  • Required: "out" (output WAV path), "text" (UTF-8 string)

  • Optional: "lang" (language code; falls back to --language), "speaker" (path to speaker JSON; falls back to --speaker)

The process loads NVIGI and the model once, then runs inference for every line. Options: --batch_skip_existing true (skip if "out" already exists), --batch_quiet true (less per-utterance logging). Lines starting with # and empty lines are ignored.

Example (multilingual, CUDA):

nvigi.tts.chatterbox.exe --models ..\..\..\data\nvigi.models --multilingual -b cuda --batch jobs.jsonl --batch_skip_existing true --batch_quiet true

If some rows omit "speaker" or "lang", supply defaults with --speaker and/or --language.

Building the Sample from Source#

While the pre-built CLI sample is ready to use, building from source allows you to:

  • Debugging: Step through the TTS pipeline in Visual Studio to understand inference flow and diagnose issues

  • Customization: Modify parameters, add batch processing, or integrate TTS into your own tooling

  • Learning: Study the code to understand NVIGI plugin loading, speaker embedding setup, and audio output handling

  • Testing: Validate fixes or test new features before integrating them into your own project

The CLI sample source is in <PACK_ROOT>/source/samples/nvigi.tts.chatterbox.

  1. Open a command prompt to <PACK_ROOT>

  2. Run .\setup_sample.bat to download the NVIGI Core SDK and set up dependencies:

    .\setup_sample.bat "C:\path\to\nvigi_developer_full_sdk"

    The NVIGI Developer SDK path is optional (not needed for TTS-only usage).

  3. Run .\setup.bat to generate the Visual Studio solution

  4. Run .\build.bat -Release (or -Production)

  5. Run .\copy_sdk_binaries.bat Release (or Production) to copy the binary to <PACK_ROOT>\bin\x64\Release

Command-line Options#

Arguments

Effect

-m, --models <path>

Path to models directory (required)

-p, --speaker <path>

Path to speaker JSON (required)

-t, --text <string>

Text prompt (required unless --interactive)

-o, --output <path>

Output WAV path (default: output.wav)

-b, --backend <cuda|vulkan|d3d12>

Backend selection (default: cuda)

-g, --guid <guid>

Model GUID (auto-selected based on --multilingual flag)

-i, --interactive

Enable interactive mode with audio playback (Windows only)

--multilingual

Use multilingual model instead of turbo (23 languages)

-l, --language <code>

Language code for multilingual (default: en). Supported: ar, da, de, el, en, es, fi, fr, he, hi, it, ja, ko, ms, nl, no, pl, pt, ru, sv, sw, tr, zh

--max_new_tokens <int>

Max tokens generated (default: 768)

--gpu_device <int>

GPU device index (default: 0)

--n_gpu_layers <int>

GPU layers for GGML/llama.cpp (default: 99)

--temperature <float>

Sampling temperature (default: 0.8)

--cfg_weight <float>

CFG weight for speech pacing (default: 0.5). Lower = slower/natural, higher = faster. Multilingual only.

--max_words_per_chunk <int>

Words per chunk, 0 disables (default: 40)

--no_perf_log

Disable CSV performance logging

Voice Cloning#

The Chatterbox TTS plugin supports zero-shot voice cloning, allowing you to create custom speaker embeddings from just a few seconds of reference audio. The voice_cloning directory contains scripts to generate speaker embedding JSON files that can be used with the TTS samples.

For complete setup instructions and usage guide, see Create Custom Voices with Voice Cloning in the Getting Started guide.

Quick Reference#

Requirements:

  • Python 3.11 (automatically installed if not found)

  • NVIDIA GPU with CUDA support (recommended) or CPU

  • 5-10 seconds of clear reference audio (WAV format)

  • Hugging Face account with access token

Basic Usage:

setup_venv.ps1 creates a virtual environment under voice_cloning\venv, installs torch, chatterbox-tts, and the rest of the dependencies into it, and activates the venv in the current PowerShell session. After that, python resolves to the venv interpreter and get_voice_embeddings.py can be invoked directly. In a new shell, re-activate the venv with .\venv\Scripts\Activate.ps1 before running the script.

cd <PACK_ROOT>\voice_cloning
.\setup_venv.ps1
# (in a new shell, first run: .\venv\Scripts\Activate.ps1)

# For Turbo model (default)
python get_voice_embeddings.py --wav reference_audio.wav --dump-json my_voice.json --hf-token YOUR_HF_TOKEN

# For Multilingual model
python get_voice_embeddings.py --wav reference_audio.wav --model-type multilingual --dump-json my_voice_multilingual.json --hf-token YOUR_HF_TOKEN

copy my_voice.json ..\data\nvigi.test\spk_emb\

Command-line Options:

Argument

Description

--wav <path>

Path to the reference WAV file (5-10 seconds recommended)

--model-type <turbo|base|multilingual>

Model type for embedding extraction (default: turbo)

--dump-json <path>

Output path for the speaker embedding JSON file

--exaggeration <float>

Emotion exaggeration baked into the embedding (default: 0.5, range: 0.0-1.0). Affects Multilingual output only; ignored by Turbo.

--hf-token <token>

Hugging Face token for model download (or use HF_TOKEN env var)

Models and Test Data#

The Chatterbox TTS pack includes:

  • Chatterbox TTS Models: Located in data/nvigi.models/nvigi.plugin.tts.chatterbox

    • Turbo (eTurbo): Fast English-only model (~2 GB VRAM)

    • Multilingual (eMultilingual): 23-language model with CFG (~3 GB VRAM). Supported languages: Arabic, Danish, German, Greek, English, Spanish, Finnish, French, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese, Russian, Swedish, Swahili, Turkish, Chinese

    • Compatible with CUDA, Vulkan, and D3D12 backends

    • Optimized for real-time text-to-speech synthesis

  • Speaker Embeddings: Located in data/nvigi.test/spk_emb/

    • JSON files containing voice profiles

    • Turbo speakers: aaron_turbo.json, lucy_turbo.json

    • Multilingual per-language speakers: arabic.json, chinese.json, danish.json, dutch.json, english.json, finnish.json, french.json, german.json, greek.json, hebrew.json, hindi.json, italian.json, japanese.json, korean.json, malay.json, norwegian.json, polish.json, portuguese.json, russian.json, spanish.json, swahili.json, swedish.json, turkish.json (23 total, one per supported language)

    • Additional multilingual voices: ethan_multilingual.json, meera_multilingual.json, speaker_base.json

    • Multilingual and Turbo embeddings are not interchangeable — use an embedding that matches the selected modelType

    • Custom speakers can be added via the voice cloning toolkit

  • Test Media: Located in data/nvigi.test/nvigi.3d/

    • 3D scene assets and media files

Troubleshooting#

Sample Won’t Launch#

Problem: Sample executable doesn’t start or crashes immediately
Solution:

  • Ensure all DLLs are present in bin/x64/Release directory

  • Check that models are present in data/nvigi.models

  • Review log files for error messages

  • Verify NVIDIA GPU drivers are up to date

  • Update to the latest Microsoft Visual C++ Redistributable

Build Errors#

Problem: Build fails with compilation errors
Solution:

  • Ensure Visual Studio 2022 is installed with C++ development tools

  • Run .\setup.bat from <PACK_ROOT> to regenerate project files

  • Clean the solution and rebuild

  • Verify that packman dependencies downloaded correctly during setup

TTS Not Working#

Problem: No audio output when using TTS
Solution:

  • Check that the TTS model is properly loaded (shown in UI)

  • Verify speaker embedding file exists and is valid JSON

  • Check audio output device settings in Windows

  • Review log files for inference errors

Voice Input Not Working#

Problem: Recording fails or no transcription
Solution:

  • Ensure a microphone is connected and set as default in Windows

  • Check Windows microphone permissions for the application

  • Verify ASR model is loaded (shown in UI as “ASR: model_name”)

  • Test microphone in Windows Sound settings

GPT Not Responding#

Problem: No GPT response when submitting text
Solution:

  • Verify GPT model is loaded (shown in UI as “GPT: model_name”)

  • Check the nvigi.model.config.json exists in the GPT model directory

  • Review logs for GPT inference errors

  • Ensure sufficient VRAM is available

Performance Issues#

Problem: Slow inference or frame rate drops
Solution:

  • Adjust GPU scheduling mode in App Settings (Prioritize Inference vs Graphics)

  • Enable frame rate limiter to reduce GPU contention

  • Ensure no other applications are using excessive GPU resources

  • Use the Turbo model variant for faster inference

  • Check that CUDA in Graphics (CIG) is enabled (D3D12 mode)

Vulkan Mode Issues#

Problem: Vulkan mode crashes or doesn’t work
Solution:

  • Ensure GPU supports Vulkan

  • Update GPU drivers to latest version

  • Check logs for Vulkan-specific errors

  • Try D3D12 mode as an alternative

D3D12 Mode Issues#

Problem: D3D12 plugins not available or showing as unavailable in UI
Solution:

  • D3D12 plugins require NVIDIA driver version 580.61 or newer

  • Update GPU drivers to the latest version

  • If D3D12 plugins are unavailable, the sample will automatically fall back to CUDA backends

  • Check logs for driver version warnings or compatibility issues

Support#

For issues, questions, or feedback, please contact NVIDIA Developer Support or refer to the NVIGI Developer Pack documentation.