Is this page helpful?

Programming Guide: Automatic Speech Recognition with Nemotron ASR (GGML)#

This guide shows how to use NVIGI to integrate the Nemotron ASR GGML plugin into an application. The plugin runs offline Parakeet TDT and streaming Nemotron RNNT models on CUDA, D3D12, Vulkan, and CPU.

AUDIO FORMAT (HARD REQUIREMENT): All backends consume 16 kHz, 16-bit, mono PCM. The plugin does not resample, downmix, or convert application audio.

MODEL EXECUTION: Parakeet v2 and v3 are offline models. Nemotron Streaming EN and Streaming 3.5 require the streaming Start → Data → Stop lifecycle.

IMPORTANT: This guide contains pseudocode. For complete working references, see source/samples/nvigi.asr.sample/sample_asr.cpp and source/samples/nvigi.3d/ in the developer pack.

1.0 Initialize and Shutdown#

Load nvigi.core.framework.dll from a trusted absolute path, resolve the NVIGI entry points, and tell NVIGI where the plugin DLLs live:

#include <windows.h>
#include <nvigi.h>
#include <nvigi_asr_nemotron.h>

#include <chrono>
#include <condition_variable>
#include <cstring>
#include <mutex>
#include <string>
#include <thread>
#include <vector>

HMODULE core = LoadLibraryExA(
    (std::string(sdkPath) + "\\nvigi.core.framework.dll").c_str(),
    nullptr, LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR | LOAD_LIBRARY_SEARCH_DEFAULT_DIRS);

auto nvigiInit = (PFun_nvigiInit*)GetProcAddress(core, "nvigiInit");
auto nvigiLoadInterface =
    (PFun_nvigiLoadInterface*)GetProcAddress(core, "nvigiLoadInterface");
auto nvigiUnloadInterface =
    (PFun_nvigiUnloadInterface*)GetProcAddress(core, "nvigiUnloadInterface");
auto nvigiShutdown =
    (PFun_nvigiShutdown*)GetProcAddress(core, "nvigiShutdown");

const char* pluginPaths[] = {sdkPath};
nvigi::Preferences pref{};
pref.numPathsToPlugins = 1;
pref.utf8PathsToPlugins = pluginPaths;
pref.utf8PathToLogsAndData = logPath;

if (NVIGI_FAILED(result, nvigiInit(pref, nullptr, nvigi::kSDKVersion)))
{
    // Handle initialization failure.
}

Validate core and every resolved function before calling them. nvigiInit and nvigiShutdown are once-per-process operations. Section 9 shows the matching shutdown and FreeLibrary calls.

In non-Production builds, nvigi.core.framework.json beside the core DLL can override matching Preferences fields. Review this file before a Production deployment.

2.0 Obtain the ASR Interface#

Each backend has its own DLL and PluginID. The backend selected here must match the backend enum used when the ASR instance is created.

Backend

DLL

PluginID constant

CUDA

nvigi.plugin.asr.nemotron-ggml.cuda.dll

nvigi::plugin::asr::nemotron_ggml::cuda::kId

D3D12

nvigi.plugin.asr.nemotron-ggml.d3d12.dll

nvigi::plugin::asr::nemotron_ggml::d3d12::kId

Vulkan

nvigi.plugin.asr.nemotron-ggml.vk.dll

nvigi::plugin::asr::nemotron_ggml::vulkan::kId

CPU

nvigi.plugin.asr.nemotron-ggml.cpu.dll

nvigi::plugin::asr::nemotron_ggml::cpu::kId

The following example obtains the CUDA interface:

nvigi::IAutoSpeechRecognition* iasr{};
if (NVIGI_FAILED(result, nvigiGetInterfaceDynamic(
        nvigi::plugin::asr::nemotron_ggml::cuda::kId,
        &iasr, nvigiLoadInterface)))
{
    // Handle interface-load failure.
}

There is no automatic in-process fallback. If CUDA is unavailable, explicitly obtain the D3D12, Vulkan, or CPU interface and use its matching creation enum.

3.0 Create an ASR Instance#

Instance creation loads the GGUF, builds the compute graphs for the selected backend, and performs warmup. Create instances away from a latency-sensitive render thread.

3.1 Pick a Model#

The shipped ASR models live under data/nvigi.models/nvigi.plugin.asr.nemotron-ggml/{GUID}/.

Model

GUID

Execution

Parakeet v2

{163BD6DC-248D-4238-BC9B-B76D2AB1C3DD}

Offline

Parakeet v3

{9739C15F-9A65-484D-B091-FFA4B9DDD0EE}

Offline

Nemotron Streaming EN

{9BC5E772-850B-4F34-AE71-311F39AB438D}

Streaming

Nemotron Streaming 3.5

{36DFEC37-2FA0-4BE3-8D2F-03699FB433BE}

Streaming

To enumerate the models actually installed, leave modelGUID unset while querying capabilities:

nvigi::CommonCreationParameters common{};
common.utf8PathToModels = modelRepositoryPath;

nvigi::ASRNemotronGGMLCreationParameters asrParams{};
asrParams.chain(common);
asrParams.backend = nvigi::ASRGGMLBackend::eCUDA;

nvigi::ASRNemotronGGMLCapabilitiesAndRequirements* caps{};
nvigi::Result result = nvigi::getCapsAndRequirements(iasr, asrParams, &caps);

if (result == nvigi::kResultOk && caps &&
    caps->getVersion() >= nvigi::kStructVersion1 &&
    caps->common && caps->common->getVersion() >= nvigi::kStructVersion1 &&
    caps->common->supportedModelGUIDs)
{
    const nvigi::CommonCapabilitiesAndRequirements& models = *caps->common;
    for (size_t i = 0; i < models.numSupportedModels; ++i)
    {
        const char* name = models.supportedModelNames[i];
        const char* guid = models.supportedModelGUIDs[i];
        // Present the installed model to the application for selection.
    }
}

Supplying CommonCreationParameters::modelGUID during this query filters the result to that model instead of enumerating the repository. Capability memory remains owned by the plugin and is valid until the interface is unloaded.

3.2 Common and ASR-specific Creation Parameters#

After selecting a model, build the creation parameters. GUID braces are required.

nvigi::CommonCreationParameters common{};
common.utf8PathToModels = modelRepositoryPath;
common.modelGUID = selectedModelGUID;
common.numThreads = (int32_t)std::thread::hardware_concurrency();

nvigi::ASRNemotronGGMLCreationParameters asrParams{};
if (NVIGI_FAILED(result, asrParams.chain(common)))
{
    // Handle parameter-chain failure.
}

asrParams.backend = nvigi::ASRGGMLBackend::eCUDA;
asrParams.streamingRightContext = -1; // Use the model default.

numThreads is used by the CPU backend; GPU backends ignore it. The plugin ID from section 2 and asrParams.backend must identify the same backend.

streamingRightContext applies only to streaming RNNT models. Streaming EN accepts 0, 1, 6, or 13. Streaming 3.5 accepts 0, 3, 6, or 13. Both use 13 by default.

3.3 Add Graphics Interop When the Application Renders#

An application that already owns a D3D12 or Vulkan device can pass that graphics context to NVIGI. This avoids creating an unrelated graphics device and enables CUDA-in-Graphics (CIG) when the CUDA backend runs alongside the renderer.

ASR backend

Application renderer

Parameter to chain

Effect

CUDA

D3D12

D3D12Parameters

Create a CUDA-in-Graphics context from an application queue

CUDA

Vulkan

VulkanParameters

Create a CUDA-in-Graphics context from an application queue

D3D12

D3D12

D3D12Parameters

Reuse the application device, queues, and optional resource callbacks

Vulkan

Vulkan

VulkanParameters

Reuse the application instance, device, queues, and optional allocation callbacks

CPU

Any

None

Graphics parameters do not apply

For a D3D12 renderer, include nvigi_d3d12.h and chain the application objects before calling createInstance():

nvigi::D3D12Parameters d3d12Params{};
d3d12Params.device = d3d12Device;
d3d12Params.queue = directQueue;
d3d12Params.queueCompute = computeQueue;
d3d12Params.queueCopy = copyQueue;

if (NVIGI_FAILED(result, asrParams.chain(d3d12Params)))
{
    // Handle parameter-chain failure.
}

For a Vulkan renderer, include nvigi_vulkan.h and provide the handles owned by the application:

nvigi::VulkanParameters vulkanParams{};
vulkanParams.instance = vulkanInstance;
vulkanParams.physicalDevice = vulkanPhysicalDevice;
vulkanParams.device = vulkanDevice;
vulkanParams.queue = graphicsQueue;
vulkanParams.queueCompute = computeQueue;
vulkanParams.queueTransfer = transferQueue;

if (NVIGI_FAILED(result, asrParams.chain(vulkanParams)))
{
    // Handle parameter-chain failure.
}

The graphics objects and queues must remain valid for the ASR instance’s lifetime. Create a separate graphics-parameter object for each parameter chain; do not chain the same object into multiple instances. If the application does not render, leave these structures unchained and let the selected GPU backend create its own device.

See appendices B through E for D3D12 deployment, Vulkan device setup, CIG scheduling, and allocation callbacks.

3.4 Putting it Together#

Create the instance after all required and optional creation parameters have been chained:

nvigi::InferenceInstance* asrInstance{};
if (NVIGI_FAILED(result, iasr->createInstance(asrParams, &asrInstance)))
{
    // Handle model-load, graph-build, or warmup failure.
}

That completes the creation surface. Everything after this point is per-transcription work.

4.0 Audio Input#

Provide signed 16-bit, 16 kHz, mono PCM. Build the input data like this:

nvigi::CpuData audioData{};
audioData.buffer = pcm.data();
audioData.sizeInBytes = pcm.size() * sizeof(int16_t);

nvigi::InferenceDataAudio audio{audioData};
audio.bitsPerSample = 16;
audio.samplingRate = 16000;
audio.channels = 1;

For offline transcription, pcm contains the complete clip. For streaming, update audioData.buffer and audioData.sizeInBytes for each submitted chunk.

5.0 Receive Inferred Data#

Both offline and streaming results arrive through the inference callback. Copy output data before returning because plugin-owned slots and UTF-8 pointers are valid only during the callback.

struct HostASRCallbackCtx
{
    std::mutex mutex;
    std::condition_variable completed;
    std::string transcription;
    std::string partial;
    bool done = false;
    bool invalid = false;
};

HostASRCallbackCtx myCallbackCtx{};

auto asrCallback = [](
    const nvigi::InferenceExecutionContext* execCtx,
    nvigi::InferenceExecutionState state,
    void* userData) -> nvigi::InferenceExecutionState
{
    auto* userCtx = static_cast<HostASRCallbackCtx*>(userData);
    if (!userCtx)
        return nvigi::kInferenceExecutionStateInvalid;
    bool terminal = false;

    {
        std::lock_guard<std::mutex> lock(userCtx->mutex);

        if (execCtx && execCtx->outputs)
        {
            const nvigi::InferenceDataText* text{};
            if (execCtx->outputs->findAndValidateSlot(
                    nvigi::kASRDataSlotTranscribedText, &text) && text)
            {
                const char* utf8 = text->getUTF8Text();
                userCtx->transcription = utf8 ? utf8 : "";
            }

            const nvigi::InferenceDataText* partial{};
            if (execCtx->outputs->findAndValidateSlot(
                    nvigi::kASRDataSlotPartialText, &partial) && partial)
            {
                const char* utf8 = partial->getUTF8Text();
                userCtx->partial = utf8 ? utf8 : "";
            }
        }

        if (state == nvigi::kInferenceExecutionStateDone)
        {
            userCtx->done = true;
            terminal = true;
        }
        else if (state == nvigi::kInferenceExecutionStateInvalid)
        {
            userCtx->done = true;
            userCtx->invalid = true;
            terminal = true;
        }
    }

    if (terminal)
        userCtx->completed.notify_all();

    return state;
};

The callback states mean:

  • kInferenceExecutionStateDataPartial: tentative output that may be replaced.

  • kInferenceExecutionStateDataPending: committed output that will not change, with more output still expected.

  • kInferenceExecutionStateDone: terminal success; no more output is expected.

  • kInferenceExecutionStateInvalid: terminal failure; discard the session.

Callbacks may run on a worker thread. Keep them short, protect shared state, and never destroy an instance from inside its callback.

6.0 Prepare the Execution Context#

6.1 Runtime Parameters#

Create one ASR runtime structure per transcription:

nvigi::ASRNemotronGGMLRuntimeParameters asrRuntime{};

For Parakeet, leave the streaming-only fields at their defaults. For streaming RNNT, chain StreamingParameters and set all stream configuration before Start:

nvigi::StreamingParameters streamParams{};
streamParams.mode = nvigi::StreamingMode::eStreamingModeInputOutput;

if (NVIGI_FAILED(result, asrRuntime.chain(streamParams)))
{
    // Handle parameter-chain failure.
}

asrRuntime.languageCode = "auto";

Language prompting is used by Streaming 3.5. The shipped choices are auto, en-US, fr-FR, es-ES, de-DE, it-IT, ja-JP, ko-KR, and zh-CN. Streaming EN is English-only.

6.2 Word and Phrase Boosting#

Word and phrase boosting is also streaming-only:

nvigi::ASRWordBoostingEntry boosts[] = {
    {"Nemotron ASR", 2.0f},
    {"Nemotron", 0.0f}
};

asrRuntime.wordBoostingEntries = boosts;
asrRuntime.wordBoostingEntryCount = 2;
asrRuntime.defaultWordBoost = 1.25f;

A zero per-entry boost uses defaultWordBoost. Keep the entry array and its strings alive through Start. Language and boosting are captured at Start and must not change during an active stream.

6.3 Build the Execution Context#

Build the input slot and execution context after the callback and runtime parameters are ready:

std::vector<nvigi::InferenceDataSlot> slots = {
    {nvigi::kASRDataSlotAudio, audio}
};
nvigi::InferenceDataSlotArray inputs = {slots.size(), slots.data()};

nvigi::InferenceExecutionContext asrContext{};
asrContext.instance = asrInstance;
asrContext.callback = asrCallback;
asrContext.callbackUserData = &myCallbackCtx;
asrContext.inputs = &inputs;
asrContext.runtimeParameters = asrRuntime;

The execution context and everything it references must remain valid until evaluate() returns or the asynchronous operation reaches Done or Invalid.

7.0 Add ASR Inference to the Pipeline#

7.1 Offline Parakeet#

Submit the complete audio buffer with blocking evaluate():

if (NVIGI_FAILED(result, asrInstance->evaluate(&asrContext)))
{
    // Handle transcription failure.
}

The callback runs before evaluate() returns. Read the copied final transcription from myCallbackCtx.transcription. Do not use streaming language, boosting, VAD, or partial text with Parakeet.

7.2 Streaming Nemotron RNNT#

Streaming uses evaluateAsync() with this lifecycle:

Start -> zero or more Data calls -> Stop

The first audio chunk accompanies Start. Repeat the Data block for subsequent chunks. Stop contains no audio and requests finalization.

streamParams.signal = nvigi::StreamSignal::eStreamSignalStart;
audioData.buffer = firstChunk;
audioData.sizeInBytes = firstChunkBytes;
if (NVIGI_FAILED(result, asrInstance->evaluateAsync(&asrContext)))
{
    // Stop submitting and recreate the failed session.
}

streamParams.signal = nvigi::StreamSignal::eStreamSignalData;
audioData.buffer = nextChunk;
audioData.sizeInBytes = nextChunkBytes;
if (NVIGI_FAILED(result, asrInstance->evaluateAsync(&asrContext)))
{
    // Stop submitting and recreate the failed session.
}

streamParams.signal = nvigi::StreamSignal::eStreamSignalStop;
audioData.buffer = nullptr;
audioData.sizeInBytes = 0;
if (NVIGI_FAILED(result, asrInstance->evaluateAsync(&asrContext)))
{
    // A failed Stop does not guarantee a terminal callback.
}

Check every submission before advancing the lifecycle. Duplicate Start, Data or Stop before Start, Data after Stop, and an un-signaled streaming call are invalid.

After a successful Stop, wait for the terminal callback before cleanup:

std::unique_lock<std::mutex> lock(myCallbackCtx.mutex);
bool completed = myCallbackCtx.completed.wait_for(
    lock, std::chrono::seconds(30), [&] { return myCallbackCtx.done; });

if (!completed || myCallbackCtx.invalid)
{
    // Handle timeout or terminal inference failure.
}

The shipped sample uses chunks of 160 * 8 * (1 + streamingRightContext) samples. Applications may use their capture cadence while preserving the signal order and audio format.

8.0 Destroy the Instance#

Destroy the instance only after blocking inference returns or streaming reaches a terminal callback:

if (NVIGI_FAILED(result, iasr->destroyInstance(asrInstance)))
{
    // Check the cleanup error.
}

One inference instance represents one serialized session. Create separate instances for genuinely concurrent streams.

9.0 Unload the Interface#

Unload the same backend interface obtained in section 2, then shut down NVIGI and release the core library:

if (NVIGI_FAILED(result, nvigiUnloadInterface(
        nvigi::plugin::asr::nemotron_ggml::cuda::kId, iasr)))
{
    // Check the unload error.
}

nvigiShutdown();
FreeLibrary(core);

That completes the core API lifecycle.

Appendix#

A. Silero VAD and Speech-state Output#

Silero VAD is optional and applies only to streaming RNNT. Before instance creation, confirm that the version-2 capabilities advertise the Silero GUID:

bool sileroAvailable = false;
if (caps && caps->getVersion() >= nvigi::kStructVersion2 &&
    caps->supportsSpeechState && caps->supportedVADModelGUIDs)
{
    for (uint32_t i = 0; i < caps->supportedVADModelCount; ++i)
    {
        const char* vadGUID = caps->supportedVADModelGUIDs[i];
        if (vadGUID && std::strcmp(
                vadGUID, nvigi::kASRSileroVADModelGUID) == 0)
        {
            sileroAvailable = true;
            break;
        }
    }
}

If advertised, chain the creation parameter before createInstance():

nvigi::ASRNemotronGGMLVADCreationParameters vadCreate{};
if (NVIGI_FAILED(result, asrParams.chain(vadCreate)))
{
    // Handle parameter-chain failure.
}

Chain the runtime parameter before streaming Start:

nvigi::ASRNemotronGGMLVADRuntimeParameters vadRuntime{};
vadRuntime.mode = nvigi::ASRVADMode::eReportSpeechState;
if (NVIGI_FAILED(result, asrRuntime.chain(vadRuntime)))
{
    // Handle parameter-chain failure.
}

In the callback, validate kASRDataSlotSpeechState as InferenceDataASRSpeechState and copy its values before returning. VAD reports metadata; it does not gate, drop, reset, or end ASR audio. See Silero VAD for the full workflow.

B. D3D12 Graphics Interop#

D3D12Parameters lets the D3D12 ASR backend reuse the application’s device and queues. The CUDA backend uses the same structure to create a CUDA-in-Graphics context from a D3D12 queue.

Member

Purpose

device

Application-owned ID3D12Device

queue

Direct graphics queue

queueCompute

Asynchronous compute queue; used when the direct queue cannot create the shared CUDA context

queueCopy

Copy queue

createCommittedResourceCallback / destroyResourceCallback

Optional hooks for engine-owned allocation and accounting

flags

ReBAR, shared-queue, and compute-tuning controls

The device must support shader model 6.6 or newer. The pack uses Microsoft Agility SDK 615 and stages the following layout under bin/x64:

bin/x64/
  D3D12/
    D3D12Core.dll
    d3d12SDKLayers.dll
  dxcompiler.dll
  dxil.dll

If the application executable is staged in bin/x64, preserve that relative layout and export the matching Agility SDK selection:

extern "C" __declspec(dllexport) UINT D3D12SDKVersion = 615;
extern "C" __declspec(dllexport) const char* D3D12SDKPath = ".\\D3D12\\";

The D3D12 flags are:

Flag

Use

eDisableReBAR

Disable the GPU upload-heap path when ReBAR is unavailable or must not be used

eComputeQueueSharedWithFrame

Tell NVIGI that the supplied compute queue is also used by frame rendering

eEnableComputeAutoTuning

Tune inference compute shaders; the first iterations can be slower while tuning runs

ReBAR is an optimization, not a requirement. Leave the flags at their defaults unless the application’s device or scheduling policy requires one of these controls.

C. Vulkan Graphics Interop#

VulkanParameters lets the Vulkan ASR backend reuse an application-created Vulkan instance and device. The CUDA backend uses the queue-bearing form for Vulkan CIG.

Member

Purpose

instance

Application-owned VkInstance

physicalDevice

Physical device used to create device

device

Application-owned VkDevice

queue

Graphics-capable queue

queueCompute

Compute-only queue when the application supplies one

queueTransfer

Transfer-only queue when the application supplies one

allocateMemoryCallback / freeMemoryCallback

Optional hooks for Vulkan device-memory allocation

For CUDA-in-Graphics, supply the graphics and compute queues. For direct Vulkan-backend reuse, the current 3D sample passes instance, physicalDevice, and device; queue handles may also be supplied when the application wants NVIGI to use its queues.

The current 3D sample requests these device extensions before device creation:

  • VK_KHR_shader_integer_dot_product

  • VK_KHR_pipeline_executable_properties

  • VK_NV_shader_sm_builtins

The sample treats them as optional and lets the backend select available paths. Use source/samples/nvigi.3d/src/DeviceManagerOverride/DeviceManagerOverride_VK.cpp as the reference when adapting an existing Vulkan device manager.

D. CUDA-in-Graphics Scheduling#

When the CUDA ASR backend is created with D3D12Parameters or VulkanParameters, the application can choose how the GPU balances inference and rendering. Load the HWI Common interface and set the scheduling mode before inference:

#include <nvigi_cuda.h>
#include <nvigi_hwi_common.h>

nvigi::IHWICommon* hwiCommon{};
if (NVIGI_FAILED(result, nvigiGetInterfaceDynamic(
        nvigi::plugin::hwi::common::kId,
        &hwiCommon, nvigiLoadInterface)))
{
    // Handle interface-load failure.
}

if (NVIGI_FAILED(result, hwiCommon->SetGpuInferenceSchedulingMode(
        nvigi::SchedulingMode::kBalance)))
{
    // Handle scheduling failure.
}

Mode

Use

kPrioritizeCompute

Favor inference latency

kBalance

Balance inference and rendering; recommended starting point

kPrioritizeGraphics

Favor frame-rate-sensitive rendering

This control applies to CUDA-in-Graphics. It is not a CPU, native D3D12, or native Vulkan ASR setting. The plugin reapplies the current mode after CUDA warmup and before every inference submission, so changing the mode affects the next call.

Scheduling priority requires NVIDIA driver R575 or newer. On an older driver, applying the scheduling mode can return kResultDriverOutOfDate; this driver requirement applies to the priority control, not to ASR without CIG scheduling. Keep the HWI Common interface loaded while it is in use and unload it during the same shutdown sequence as the ASR interface. The shipped 3D sample demonstrates both D3D12 and Vulkan CIG setup in source/samples/nvigi.3d/.

E. Graphics Memory Allocation Hooks#

Applications that centralize GPU allocations can connect the ASR backend to their resource accounting or allocator. These callbacks are optional.

For D3D12, provide both callbacks as a pair:

d3d12Params.createCommittedResourceCallback = createASRResource;
d3d12Params.destroyResourceCallback = destroyASRResource;
d3d12Params.createCommitResourceUserContext = &graphicsAllocator;
d3d12Params.destroyResourceUserContext = &graphicsAllocator;

createASRResource must have the PFun_createCommittedResource signature and return an ID3D12Resource*. destroyASRResource must have the PFun_destroyResource signature and release the resource created by its paired callback.

For Vulkan, provide the allocation and free callbacks as a pair:

vulkanParams.allocateMemoryCallback = allocateASRMemory;
vulkanParams.freeMemoryCallback = freeASRMemory;
vulkanParams.allocateMemoryCallbackUserContext = &graphicsAllocator;
vulkanParams.freeMemoryCallbackUserContext = &graphicsAllocator;

allocateASRMemory receives the device, allocation size, memory-type index, and output memory handle. freeASRMemory receives the device and the memory handle returned by the allocator. Keep each callback and its user context valid until the ASR instance is destroyed.

F. Threading and Lifetime#

  • Keep all pointer-backed audio and runtime data valid until the plugin consumes it.

  • Do not mutate language, boosting, VAD, or right-context state during a stream.

  • Do not destroy an instance from inside its callback.

  • Do not unload the interface while instances or callbacks remain active.

  • Use one in-flight transcription per instance.

G. Unsupported or Family-specific Behavior#

  • Beam search/KenLM and a separate punctuation stage are not exposed.

  • Parakeet does not expose streaming partial text, right context, language prompting, boosting, or Silero speech state.

  • Streaming RNNT does not accept an offline un-signaled call.

  • Successful execution does not establish transcription quality or WER.

H. Troubleshooting#

#

Symptom

Likely cause

Fix

1

Model is not advertised

Repository path, GUID directory, GGUF, or config is incorrect

Verify the repository and exact brace-wrapped GUID

2

Plugin DLL fails to load

Backend ID, DLL, or dependency mismatch

Use the matching backend row from section 2

3

evaluate returns an invalid parameter

Audio is not 16 kHz, 16-bit, mono PCM

Convert audio before submission

4

Streaming returns invalid state

Incorrect Start/Data/Stop ordering

Start once, submit Data, then Stop

5

Right context is rejected

Value is not supported by the selected streaming profile

Streaming EN: 0/1/6/13; Streaming 3.5: 0/3/6/13

6

No partial text

An offline model was selected or partial callbacks were ignored

Use streaming RNNT and handle DataPartial/DataPending

7

VAD is unavailable

Capability version, speech-state support, or Silero GUID check failed

Complete the checks in appendix A

8

Language or boosting change is ignored

The value changed after Start

Set it before the next Start

9

D3D12 backend fails during startup

Agility SDK or DXC files were moved out of their required layout

Preserve D3D12/, dxcompiler.dll, and dxil.dll as shown in appendix B

10

CUDA CIG instance creation fails

The supplied graphics queue could not create a shared context and no usable compute queue was supplied

Provide the renderer’s compute queue and follow the 3D sample CIG setup

11

Vulkan device reuse fails

Instance, physical device, device, queues, or enabled features do not describe one compatible Vulkan device

Build the parameter set from one device and compare device creation with the 3D sample