Programming Guide: Qwen3-TTS with NVIGI#

Scope#

This guide describes the NVIGI Qwen3-TTS plugin shipped in the Qwen TTS pack. It covers direct text-to-speech and voice cloning through CUDA, Vulkan, and D3D12 backends.

The standalone Qwen pack does not include ASR or GPT plugins/models. For a full ASR -> GPT -> TTS integration test, run the following from the pack root with a compatible prebuilt NVIGI Developer SDK:

setup_sample.bat "C:\path\to\nvigi_developer_full_sdk"

This stages the ASR/GPT plugins and downloads their model weights into the pack before running the 3D sample.

Runtime Components#

Component	Purpose
`nvigi.plugin.tts.qwen3.cuda.dll`	Qwen3-TTS on CUDA
`nvigi.plugin.tts.qwen3.vk.dll`	Qwen3-TTS on Vulkan
`nvigi.plugin.tts.qwen3.d3d12.dll`	Qwen3-TTS on D3D12
`qwen-talker-0.6b-base-Q4_K_M.gguf`	Default Qwen Talker model
`qwen-talker-0.6b-base-Q8_0.gguf`	Optional higher-precision Qwen Talker
`qwen-tokenizer-12hz-F32.gguf`	Qwen codec model
`nvigi_tts_qwen3.h`	Public C++ API

The shipped model GUIDs are:

Q4_K_M: {96F1AC08-B98E-41B0-B8E4-D8DC446CF868}
Q8_0:   {A7D3620F-ADDB-4200-8EB4-3A5E270846CB}

The plugin returns 24 kHz mono PCM audio through streaming callbacks.

Supported Languages#

Pass one of these exact language names in Qwen3TTSRuntimeParameters::language:

Chinese, English, Japanese, Korean, German, French, Russian,
Portuguese, Spanish, Italian

Plugin IDs#

#include <nvigi.h>
#include <nvigi_ai.h>
#include "nvigi_tts_qwen3.h"

const nvigi::PluginID cudaId   = nvigi::plugin::tts::qwen3::cuda::kId;
const nvigi::PluginID vulkanId = nvigi::plugin::tts::qwen3::vulkan::kId;
const nvigi::PluginID d3d12Id  = nvigi::plugin::tts::qwen3::d3d12::kId;

Use a plugin ID matching the DLL deployed with the application and the intended backend.

Initialize NVIGI and Load the Interface#

The host must initialize NVIGI with the directory containing the plugin DLLs, then load one Qwen3-TTS interface.

const char* pluginPaths[] = { packageBinDir };

nvigi::Preferences preferences{};
preferences.logLevel = nvigi::LogLevel::eDefault;
preferences.numPathsToPlugins = 1;
preferences.utf8PathsToPlugins = pluginPaths;
preferences.utf8PathToLogsAndData = packageBinDir;

nvigi::Result result{};
if (NVIGI_FAILED(result, nvigiInit(preferences, nullptr, nvigi::kSDKVersion))) {
    // Handle initialization failure.
}

nvigi::ITextToSpeech* tts = nullptr;
if (NVIGI_FAILED(result, nvigiGetInterfaceDynamic(
        nvigi::plugin::tts::qwen3::cuda::kId, &tts, nvigiLoadInterface))) {
    // Handle missing or incompatible CUDA Qwen plugin.
}

On Windows, a host executable outside the package DLL directory must make that directory available to Windows DLL loading before initializing NVIGI. The shipped CLI demonstrates the expected colocated deployment.

Create a Qwen3-TTS Instance#

The simplest setup provides the model root and shipped GUID. The plugin locates the talker and codec GGUF files beneath the GUID directory.

nvigi::CommonCreationParameters common{};
common.utf8PathToModels = modelRoot; // ...\data\nvigi.models
common.modelGUID = "{96F1AC08-B98E-41B0-B8E4-D8DC446CF868}"; // Q4_K_M
common.vramBudgetMB = 2048;

nvigi::Qwen3TTSCreationParameters creation{};
if (NVIGI_FAILED(result, creation.chain(common))) {
    // Handle parameter-chain failure.
}

nvigi::InferenceInstance* instance = nullptr;
if (NVIGI_FAILED(result, tts->createInstance(creation, &instance))) {
    // Verify the model path, GUID, GGUF files, selected backend, and VRAM budget.
}

To select Q8_0, set common.modelGUID to {A7D3620F-ADDB-4200-8EB4-3A5E270846CB}. The CLI exposes the same choice through --guid; Q4_K_M is its default.

Alternatively, set both Qwen3TTSCreationParameters::talkerPath and codecPath to explicit GGUF file paths. Supplying only one direct path is invalid. common.modelGUID remains required, and the Talker filename must match that GUID. Q8_0 uses more disk/VRAM and can be slower; no backend or GPU auto-selection occurs.

creation.enableVoiceClone is reserved as a voice-cloning hint. The current Qwen adapter does not use it to reduce model loading, so applications must not rely on it for a VRAM or feature change.

D3D12 hosts#

For the D3D12 backend, chain valid nvigi::D3D12Parameters with the D3D12 device, compute queue, and required allocation callbacks before calling createInstance. The device and queues must remain valid for the instance lifetime.

The first D3D12 synthesis can include one-time backend, shader, and graph warm-up work. Its duration depends on the GPU, driver, model, and cache state; do not treat that cold-start latency as steady-state TTS performance. Applications that need a responsive first user request should run a short, non-user-visible dummy synthesis during their loading phase, then discard its audio.

The complete implementation is in:

source/samples/nvigi.tts.qwen/sample_tts_qwen.cpp

Run Streaming Inference#

Provide text through kQwen3TTSDataSlotPrompt. Audio is emitted through kQwen3TTSDataSlotGeneratedAudio.

const char* text = "Hello from Qwen3 TTS.";
nvigi::CpuData textData{};
textData.buffer = const_cast<char*>(text);
textData.sizeInBytes = strlen(text) + 1;

nvigi::InferenceDataText prompt{textData};
nvigi::InferenceDataSlot inputSlot{
    nvigi::kQwen3TTSDataSlotPrompt, prompt
};
nvigi::InferenceDataSlotArray inputs{1, &inputSlot};

nvigi::Qwen3TTSRuntimeParameters runtime{};
runtime.language = "English";
runtime.temperature = 0.9f;
runtime.topK = 50;
runtime.seed = -1; // Default: choose a fresh random seed.
runtime.maxNewTokens = 2048;
runtime.maxWordsPerChunk = 40; // Split long text at sentence/word boundaries.

auto callback = [](const nvigi::InferenceExecutionContext* context,
                   nvigi::InferenceExecutionState state,
                   void*) {
    if ((state == nvigi::kInferenceExecutionStateDataPending ||
         state == nvigi::kInferenceExecutionStateDone) &&
        context && context->outputs) {
        const nvigi::InferenceDataAudio* audio = nullptr;
        if (context->outputs->findAndValidateSlot(
                nvigi::kQwen3TTSDataSlotGeneratedAudio, &audio) && audio) {
            // Copy audio->audio.buffer now and enqueue it for playback.
            // The plugin can reuse this memory after the callback returns.
        }
    }
    return state;
};

nvigi::InferenceExecutionContext execution{};
execution.instance = instance;
execution.inputs = &inputs;
execution.runtimeParameters = runtime;
execution.callback = callback;

if (NVIGI_FAILED(result, instance->evaluate(&execution))) {
    // Handle inference failure.
}

Process both kInferenceExecutionStateDataPending and kInferenceExecutionStateDone. Handling only Done can discard earlier chunks of a long response.

Voice Cloning#

Reference-WAV workflow#

Set a mono 24 kHz WAV path. Supplying a transcript enables ICL mode B; leaving refText null uses x-vector-only mode A.

runtime.refAudioPath = "C:\\voices\\speaker.wav";
runtime.refText = "Accurate transcript of the reference recording.";

Pre-computed embedding workflow#

The pack CLI accepts a voice JSON through --speaker. For direct SDK integration, decode the JSON in the host and provide its contents through the runtime structure:

runtime.refSpkEmb = speakerEmbedding.data();
runtime.refSpkDim = static_cast<int>(speakerEmbedding.size());
runtime.refCodes = referenceCodes.data(); // optional ICL code matrix
runtime.refT = referenceFrameCount;
runtime.refText = transcript;             // required when using ICL codes

refAudioPath and refSpkEmb are mutually exclusive. Supplying both is an invalid parameter error.

To generate a compatible JSON without writing host-side extraction code, use the shipped Qwen CLI. This workflow has no Python or ML-package dependency:

bin\x64\Release\nvigi.tts.qwen.exe ^
  --models data\nvigi.models ^
  --backend cuda ^
  --refAudio C:\voices\speaker.wav ^
  --refText @C:\voices\speaker.txt ^
  --extractEmbedding C:\voices\speaker.json

Use approved and consented reference audio. The package does not include Qwen’s example WAV or a JSON derived from it. The opt-in voice_cloning\provision_qwen_example_voice.ps1 script can retrieve Qwen’s hosted example and generate a reusable JSON locally. For another approved WAV, use the CLI command above with its exact transcript.

Runtime Parameters#

Field	Default	Meaning
`language`	`English`	One supported Qwen language name
`temperature`	`0.9`	Finite, non-negative sampling temperature; `0` disables sampling
`subtalkerTemperature`	`0.9`	Finite, non-negative Code Predictor sampling temperature; `0` disables sampling
`topK`	`50`	Top-K sampling bound
`seed`	`-1`	Sampling seed; `-1` requests a fresh random seed
`maxNewTokens`	`2048`	Generation limit
`maxWordsPerChunk`	`0`	`0` keeps one request. A positive value creates bounded independent requests; provide `refAudioPath` or `refSpkEmb`/`refSpkDim` so every segment uses the same Qwen voice.
`refAudioPath`	null	Mono 24 kHz reference WAV
`refText`	null	Reference transcript; enables ICL mode B
`refSpkEmb`, `refSpkDim`	null, `0`	Pre-computed speaker embedding
`refCodes`, `refT`	null, `0`	Optional ICL reference-code matrix
`dumpDir`	null	Debug tensor dump directory; do not use in normal production synthesis

Use a non-negative seed value when a host needs controlled repeated synthesis on the same build, model, and backend. The default -1 requests a fresh random seed.

gpu_device and n_gpu_layers remain producer-owned. maxWordsPerChunk is applied by the shared TTS adapter: 0 preserves one-request behavior, while a positive value packs adjacent complete sentences up to the requested word limit and splits only when necessary. Because each resulting Qwen request begins a new generation, bounded long-form synthesis requires a Qwen voice reference (refAudioPath or refSpkEmb/refSpkDim); otherwise the adapter returns an actionable invalid-parameter error rather than concatenating unconditioned voices. The shipped CLI defaults this option to 40.

Cleanup#

Destroy the instance before unloading the interface or shutting down NVIGI:

tts->destroyInstance(instance);
nvigiUnloadInterface(
    nvigi::plugin::tts::qwen3::cuda::kId, tts);
nvigiShutdown();

Troubleshooting#

Plugin does not load: verify the selected Qwen backend DLL is deployed beside the NVIGI runtime DLLs.
Instance creation fails: verify the model root, GUID, talker GGUF, codec GGUF, and VRAM budget.
No voice-cloning result: verify the WAV is mono 24 kHz and that only one of refAudioPath or refSpkEmb is set.
Long output is truncated: copy audio on every DataPending callback, not just Done. For long prompts, set maxWordsPerChunk to a positive value (the shipped CLI defaults to 40) so each bounded segment reaches its own model EOS before the generation cap.
D3D12 creation fails: verify the D3D12 device, compute queue, and allocation callbacks remain valid.

For complete CLI behavior, run:

nvigi.tts.qwen.exe --help