Is this page helpful?

Chatterbox TTS Developer Pack Overview#

1. What’s Included#

The Chatterbox TTS Plugin Pack provides text-to-speech (TTS) capabilities for NVIDIA In-Game Inferencing (NVIGI) applications. This pack includes:

  • Pre-built TTS plugin DLLs for CUDA, Vulkan, and D3D12 backends (Production and Release configurations)

  • Chatterbox Turbo (English) and Chatterbox Multilingual (23 languages) models with bundled speaker embeddings

  • Command-line and 3D sample applications

  • Complete source code for samples (rebuild from source)

  • Voice cloning toolkit with embedded Python environment

  • Built HTML documentation

2. Text-to-Speech Models#

2.1 Included Models#

Two Chatterbox models are bundled in the pack and ready to use without any additional downloads.

Plugin

Model Name

GUID

Language(s)

nvigi.plugin.tts.chatterbox

Chatterbox Turbo

{019BD494-0D97-7223-B9D5-C9286933B8B7}

English

nvigi.plugin.tts.chatterbox

Chatterbox Multilingual

{A60EB5CF-9551-4B86-865B-CDC3CDBE61C4}

Arabic, Danish, German, Greek, English, Spanish, Finnish, French, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese, Russian, Swedish, Swahili, Turkish, Chinese

2.2 Model Components#

Turbo ({019BD494-...}/)#

File

Size

Purpose

t3_turbo_text_emb.gguf

223.74 MB

T3 text encoder — converts text to speech tokens

turbo_gpt2_backbone_v2.gguf

246.71 MB

GPT-2 backbone — transformer for speech token generation

S3Gen-Meanflow-266M-F32.gguf

508.86 MB

S3Gen vocoder — converts speech tokens to audio waveform

nvigi.model.config.json

< 1 KB

Model configuration metadata

Total Turbo model size: ~979 MB

Multilingual ({A60EB5CF-...}/)#

File

Purpose

multilingual_t3_text_emb.gguf

T3 text encoder with multilingual tokenizer

multilingual_chatterbox_llama_backbone_tts_q4_0.gguf

LLaMA 520M backbone (Q4_0 quantized) for speech token generation

S3Gen-Multilingual-F16.gguf

S3Gen vocoder

cangjie_mapping.json

Chinese Cangjie encoding table (auto-discovered for zh)

kanji_readings.json

Kanji → Hiragana reading table (auto-discovered for ja)

dicta-bert-q8.gguf / dicta-bert-q8-heads.gguf

DictaBERT diacritization model (auto-discovered for he)

nvigi.model.config.json

Model configuration metadata

The auxiliary JSON and GGUF assets are loaded automatically by the plugin when the corresponding language is selected; no additional configuration is required.

2.3 Speaker Embeddings#

Pre-extracted speaker embeddings are included in data/nvigi.test/spk_emb/:

File

Model Type

Description

aaron_turbo.json

Turbo

Male English voice

lucy_turbo.json

Turbo

Female English voice

arabic.json, chinese.json, danish.json, dutch.json, english.json, finnish.json, french.json, german.json, greek.json, hebrew.json, hindi.json, italian.json, japanese.json, korean.json, malay.json, norwegian.json, polish.json, portuguese.json, russian.json, spanish.json, swahili.json, swedish.json, turkish.json

Multilingual

Per-language reference voices (23 total, one per supported language)

ethan_multilingual.json, meera_multilingual.json, speaker_base.json

Multilingual

Additional multilingual voices

Turbo and Multilingual embeddings are not interchangeable — use one that matches the selected modelType at inference time.

Custom speaker embeddings can be created from reference audio using the voice cloning toolkit in the voice_cloning/ directory. See the Getting Started Guide for instructions.

2.4 Model Capabilities#

  • High-Quality Speech: Natural prosody and rhythm using the Chatterbox Turbo architecture (350M parameters) or Chatterbox Multilingual architecture (520M LLaMA backbone)

  • Zero-Shot Voice Cloning: Clone any voice from 5–10 seconds of reference audio (each speaker JSON is bound to either Turbo or Multilingual and is not interchangeable)

  • Paralinguistic Tags (Turbo only): Built-in support for [laugh], [cough], [sigh], [gasp], [chuckle], [groan], [sniff], [shush], [clear throat]. The Multilingual model’s vocabulary does not contain these tags — they will not produce the intended non-speech sound and should not be used with eMultilingual.

  • CFG-Controlled Pacing (Multilingual only): Runtime cfg_weight parameter for tunable speech pacing. The Turbo model ignores cfg_weight and runs without classifier-free guidance.

  • Native Language Preprocessing (Multilingual only): Chinese Cangjie encoding, Japanese Kanji→Hiragana conversion, Korean Jamo decomposition, Unicode NFKD normalization, and Hebrew diacritization performed inside the plugin (auxiliary assets auto-discovered from the model directory).

  • Hallucination Protection (Multilingual only): Built-in alignment analyzer that suppresses premature EOS and forces termination on degenerate / runaway generations.

  • Long-Text Handling: Automatic text chunking for extended prompts (word-based by default; switches to character-based for Multilingual zh / ja).

  • Real-Time Performance: Faster-than-realtime inference on supported GPUs (Turbo is ~2× faster than Multilingual)

  • Multi-Backend: CUDA, Vulkan, or D3D12 inference backends

  • Output Format: 24 kHz, 16-bit mono PCM audio

For the full per-feature breakdown of what is exclusive to Turbo vs Multilingual, see Model Feature Comparison in the Programming Guide.

3. Important Documentation in the Developer Pack#

The main documentation for using the pack includes:

  • Getting Started Guide

    • Setup, first run, voice cloning, and integration overview

  • Samples

    • The best “starting” doc, as the samples are provided pre-compiled and ready-to-run. Describes how to run, recompile, and debug the samples.

  • Chatterbox TTS Programming Guide

    • Detailed documentation on how to program for the Chatterbox TTS plugin, including plugin initialization, inference, text chunking, and API usage

  • Model Card

    • Model architecture, training data, capabilities, limitations, and license information

4. Samples Included#

The pack includes pre-built sample applications (in both Production and Release configurations) with full source code:

4.1 Command-Line Sample#

  • nvigi.tts.chatterbox — Text-to-speech CLI demo

    • Text-to-speech with customizable parameters

    • Multiple backend support (CUDA, Vulkan, D3D12)

    • WAV file output and interactive mode with real-time audio playback

    • Source: source/samples/nvigi.tts.chatterbox/

4.2 3D GUI Sample#

  • nvigi.3d — Interactive 3D application with TTS integration

    • D3D12 and Vulkan rendering with TTS audio playback

    • Real-time audio playback with GPU scheduling controls

    • Speaker selection and backend switching via GUI

    • Source: source/samples/nvigi.3d/

5. Contents of the Developer Pack#

The pack is organized as follows:

_chatterbox_tts_pack/
├── bin/x64/
│   ├── Production/                                # Production build
│   │   ├── nvigi.plugin.tts.chatterbox.cuda.dll      # TTS plugin (CUDA)
│   │   ├── nvigi.plugin.tts.chatterbox.d3d12.dll     # TTS plugin (D3D12)
│   │   ├── nvigi.plugin.tts.chatterbox.vk.dll        # TTS plugin (Vulkan)
│   │   ├── nvigi.tts.chatterbox.exe                  # CLI sample
│   │   ├── nvigi.3d.exe                              # 3D GUI sample
│   │   ├── cublas64_12.dll, cublasLt64_12.dll        # CUDA libraries
│   │   ├── cudart64_12.dll                           # CUDA runtime
│   │   ├── dxcompiler.dll, dxil.dll                  # DXC (D3D12 backend)
│   │   ├── D3D12/                                    # D3D12 Agility SDK
│   │   │   ├── D3D12Core.dll
│   │   │   └── d3d12SDKLayers.dll
│   │   └── shaders/                                  # Pre-compiled shaders (3D sample)
│   │
│   └── Release/                                   # Release build (same layout)
│       └── ...
│
├── include/                                    # NVIGI SDK headers
│   └── nvigi_tts_chatterbox.h                     # TTS plugin API header
│
├── source/                                     # Source code
│   └── samples/
│       ├── nvigi.tts.chatterbox/                  # CLI sample source
│       │   ├── sample_tts_chatterbox.cpp
│       │   └── premake.lua
│       └── nvigi.3d/                              # 3D sample source
│           ├── premake.lua
│           └── src/
│               ├── main.cpp
│               ├── NVIGISample.cpp / .h
│               └── nvigi/                         # NVIGI integration code
│                   ├── NVIGIContext.cpp / .h
│                   ├── AudioPlayer.h
│                   └── AudioRecorder.cpp / .h
│
├── data/
│   ├── nvigi.models/                           # TTS models (bundled)
│   │   └── nvigi.plugin.tts.chatterbox/
│   │       ├── {019BD494-...}/                    # Chatterbox Turbo
│   │       │   ├── t3_turbo_text_emb.gguf
│   │       │   ├── turbo_gpt2_backbone_v2.gguf
│   │       │   ├── S3Gen-Meanflow-266M-F32.gguf
│   │       │   └── nvigi.model.config.json
│   │       └── {A60EB5CF-...}/                    # Chatterbox Multilingual
│   │           ├── multilingual_t3_text_emb.gguf
│   │           ├── multilingual_chatterbox_llama_backbone_tts_q4_0.gguf
│   │           ├── S3Gen-Multilingual-F16.gguf
│   │           ├── cangjie_mapping.json           # Chinese (zh) preprocessing
│   │           ├── kanji_readings.json            # Japanese (ja) preprocessing
│   │           ├── dicta-bert-q8.gguf             # Hebrew (he) diacritization
│   │           ├── dicta-bert-q8-heads.gguf
│   │           └── nvigi.model.config.json
│   │
│   └── nvigi.test/
│       ├── spk_emb/                               # Speaker embeddings (bundled)
│       │   ├── aaron_turbo.json, lucy_turbo.json       # Turbo
│       │   ├── arabic.json, chinese.json, ...          # Multilingual (per-language)
│       │   └── ethan_multilingual.json, meera_multilingual.json, speaker_base.json
│       └── nvigi.3d/                              # 3D sample assets
│           ├── nvidia-logo.png
│           ├── fonts/
│           └── glTF-Sample-Models/
│
├── voice_cloning/                              # Voice cloning toolkit
│   ├── get_voice_embeddings.py                    # Embedding extraction script
│   ├── setup_venv.ps1                             # Environment setup (creates voice_cloning\venv)
│   ├── requirements.txt                           # Python dependencies
│   ├── female.wav                                 # Example reference audio
│   ├── speaker_base.json                          # Example output embedding (multilingual)
│   └── python311/                                 # Embedded bootstrap Python
│
├── symbols/                                    # Debug symbols (PDB files)
│   ├── nvigi.plugin.tts.chatterbox.cuda.pdb
│   ├── nvigi.plugin.tts.chatterbox.d3d12.pdb
│   ├── nvigi.plugin.tts.chatterbox.vk.pdb
│   ├── nvigi.core.framework.pdb
│   ├── nvigi.tts.chatterbox.pdb
│   ├── nvigi.3d.pdb
│   └── Production/                                # Production config PDBs
│       └── ...
│
├── docs/                                       # Built HTML documentation
│   ├── index.html
│   ├── getting-started.html
│   ├── samples.html
│   ├── programming-guide-tts-chatterbox.html
│   ├── nvigi-chatterbox-model-card.html
│   ├── NVIGI_CHATTERBOX_MODEL_CARD.pdf
│   ├── 3rd-party-licenses.html
│   └── _static/                                   # Sphinx theme assets
│
├── premake/                                    # Build system modules
│   ├── premake.shared_config.lua
│   ├── premake.shared_plugin.lua
│   ├── premake.cuda_utils.lua
│   └── premake.utils.lua
│
├── scripts/                                    # Configuration
│   └── nvigi.core.framework.json
│
├── tools/                                      # Build tools
│   ├── vswhere.exe
│   └── packman/                                   # Packman package manager
│
├── setup_sample.bat                            # Downloads NVIGI Core SDK
├── setup.bat                                   # Generates Visual Studio solution
├── build.bat                                   # Builds all projects
├── copy_sdk_binaries.bat                       # Copies built binaries to bin/x64/
├── premake.lua                                 # Root build configuration
├── project.xml                                 # Project dependencies
├── dll-info.txt                                # DLL inventory reference
├── manifest.json                               # Pack manifest
├── LICENSE.txt                                 # License
├── NOTICE.txt                                  # Notices
└── README.html                                 # Quick start instructions

6. Runtime Dependencies#

This section describes the DLLs required to run the TTS plugin and sample applications. The pack ships two build configurations under bin/x64/: Production (optimized, no debug info) and Release (optimized with debug info).

Game developers integrating TTS should focus on Plugin Dependencies.

6.1 Plugin Dependencies#

These DLLs are required when integrating the NVIGI Chatterbox TTS plugin into your game or application.

NVIGI Core Framework (from NVIGI Core SDK)#

The core framework DLLs are not included in this pack. They are downloaded by running setup_sample.bat, which fetches the NVIGI Core SDK. When integrating into your own application, obtain these from the NVIGI Core SDK:

DLL

Purpose

nvigi.core.framework.dll

Core NVIGI plugin framework — loads and manages plugins

nvigi.plugin.hwi.common.dll

Common hardware interface — shared GPU utilities

nvigi.plugin.hwi.cuda.dll

CUDA hardware interface — required for CUDA backend

nvigi.plugin.hwi.d3d12.dll

D3D12 hardware interface — required for D3D12 backend

TTS Plugin (choose one based on backend)#

DLL

Size

Purpose

nvigi.plugin.tts.chatterbox.cuda.dll

59.50 MB

CUDA backend — best performance on NVIDIA GPUs

nvigi.plugin.tts.chatterbox.d3d12.dll

32.88 MB

D3D12 backend — Windows D3D12-capable GPUs

nvigi.plugin.tts.chatterbox.vk.dll

47.01 MB

Vulkan backend — any Vulkan-capable GPU

Each TTS plugin DLL includes the GGML/llama.cpp inference engine statically linked. Only one plugin DLL is needed at runtime based on your chosen backend.

CUDA Libraries (required for CUDA backend)#

DLL

Size

Purpose

cublas64_12.dll

108.44 MB

cuBLAS — matrix operations for inference

cublasLt64_12.dll

660.36 MB

cuBLAS Lt — lightweight matrix operations

cudart64_12.dll

0.55 MB

CUDA 12 runtime

D3D12 / DXC Libraries (required for D3D12 backend)#

DLL

Size

Purpose

dxcompiler.dll

17.01 MB

DirectX Shader Compiler — compiles compute shaders for GGML D3D12

dxil.dll

1.42 MB

DXIL signing library — required by dxcompiler

System Dependencies (not shipped — expected on target system)#

DLL

Source

Required For

d3d12.dll

Windows 10+

D3D12 backend

dxgi.dll

Windows 10+

GPU enumeration, device creation

nvcuda.dll

NVIDIA Driver

CUDA backend

6.2 Application Dependencies#

These additional DLLs are required only for the sample applications. All Plugin Dependencies above also apply.

3D Sample (nvigi.3d.exe)#

DLL

Size

Location

Purpose

D3D12Core.dll

3.20 MB

bin/x64/{config}/D3D12/

D3D12 Agility SDK — provides consistent D3D12 features

d3d12SDKLayers.dll

4.84 MB

bin/x64/{config}/D3D12/

D3D12 validation/debug layers

6.3 Dependency Hierarchy#

TTS Plugin (CUDA)
├── NVIGI Core Framework (from NVIGI Core SDK)
│   ├── nvigi.core.framework.dll
│   ├── nvigi.plugin.hwi.common.dll
│   ├── nvigi.plugin.hwi.cuda.dll ──→ [nvcuda.dll, cudart64_12.dll]
│   └── nvigi.plugin.hwi.d3d12.dll ──→ [d3d12.dll, dxgi.dll]
│
├── CUDA Libraries (shipped in pack)
│   ├── cublas64_12.dll ──→ [nvcuda.dll]
│   ├── cublasLt64_12.dll ──→ [nvcuda.dll]
│   └── cudart64_12.dll ──→ [nvcuda.dll]
│
└── CIG Scheduler (optional, from NVIGI Core SDK)
    ├── cig_scheduler_settings.dll
    └── cupti64_*.dll

TTS Plugin (D3D12)
├── NVIGI Core Framework (from NVIGI Core SDK)
│
└── DXC Libraries (shipped in pack)
    ├── dxcompiler.dll ──→ [d3d12.dll]
    └── dxil.dll

TTS Plugin (Vulkan)
├── NVIGI Core Framework (from NVIGI Core SDK)
│
└── (no additional DLLs — Vulkan support is statically linked)