Is this page helpful?

Nemotron ASR Developer Pack Overview#

1. What’s Included#

The Nemotron ASR Plugin Pack provides offline and streaming speech recognition through NVIDIA In-Game Inferencing (NVIGI). This release is built on GGML as the inference engine, and ships two offline models and two streaming models.

Component

Included

ASR backends

CPU, CUDA, Vulkan, and D3D12 plugin DLLs

Offline ASR

Parakeet-TDT 0.6B v2 and v3

Streaming ASR

Nemotron Speech Streaming EN and Nemotron 3.5 ASR Streaming

Voice activity detection

Silero VAD v6.2 speech-state reporting

Samples

Command-line sample and interactive 3D sample, with source

Model tools

NeMo conversion, GGUF quantization, and backend-load verification

Integration files

Public Nemotron ASR header and NVIGI Core headers/libraries

Current integration identifiers:

Item

Value

Plugin basename

nvigi.plugin.asr.nemotron-ggml

Public header

include/nvigi_asr_nemotron.h

C++ namespace

nvigi::plugin::asr::nemotron_ggml

The same model repository is shared by every backend. Applications switch execution paths by loading a different backend DLL; models do not require per-backend conversion.

2. Automatic Speech Recognition Models#

The pack covers two offline workflows and two streaming workflows. Choose Parakeet when the complete recording is available before inference. Choose a Nemotron streaming model for live audio or word boosting functionality.

2.1 Included models#

Each ASR model is a GGUF file stored beside nvigi.model.config.json in its GUID directory. The tabs below summarize the selection information needed by an application; sizes are binary MiB.

Property

Value

Model

Parakeet-TDT 0.6B v2

GUID

{163BD6DC-248D-4238-BC9B-B76D2AB1C3DD}

Mode / decoder

Offline / TDT

Language metadata

en-US

GGUF size

363.29 MiB

Invocation

Complete buffer with evaluate() or evaluateAsync()

Property

Value

Model

Parakeet-TDT 0.6B v3

GUID

{9739C15F-9A65-484D-B091-FFA4B9DDD0EE}

Mode / decoder

Offline / TDT

Language metadata

multilingual

GGUF size

389.90 MiB

Invocation

Complete buffer with evaluate() or evaluateAsync()

Property

Value

Model

Nemotron Speech Streaming EN 0.6B RNNT

GUID

{9BC5E772-850B-4F34-AE71-311F39AB438D}

Mode / decoder

Streaming / RNNT

Language metadata

en-US

GGUF size

377.64 MiB

Right context

0, 1, 6, or 13; default 13

Property

Value

Model

Nemotron 3.5 ASR Streaming 0.6B RNNT

GUID

{36DFEC37-2FA0-4BE3-8D2F-03699FB433BE}

Mode / decoder

Streaming / RNNT

Language metadata

multi

GGUF size

417.59 MiB

Right context

0, 3, 6, or 13; default 13

Property

Value

Model

Silero VAD v6.2

GUID

{8A5C059F-EDCA-49D1-875D-4F6CF208E5CE}

Role

Optional companion for streaming models

GGUF size

1.18 MiB

Output

Speech state, probability, and processed-sample count

Silero is advertised through the VAD capability fields; it is not an ASR acoustic model.

2.2 Capability matrix#

Model families expose different runtime contracts. Streaming controls are not global plugin settings and must not be applied to the offline Parakeet models.

Capability

Parakeet v2/v3

Streaming EN

Streaming 3.5

Live partial text

No

Yes

Yes

Start/Data/Stop streaming

No

Yes

Yes

Language prompt

No

No

Yes; default auto

Word/phrase boosting

No

Yes

Yes

Optional Silero speech state

No

Yes

Yes

3. Accuracy and Performance#

These results compare the four shipped ASR models across every packaged backend. Use them to choose a starting backend and model, then measure the target application’s audio, hardware, concurrency, and latency requirements independently.

3.1 Test configuration#

All rows use the same English evaluation set and text normalization. Streaming models were measured with their RC13 profile so the backend comparison remains consistent.

Item

Configuration

Dataset

LibriSpeech test-clean, 2,620 samples per run

Backends

CPU, CUDA, Vulkan, D3D12

GPU

NVIDIA GeForce RTX 4090; driver 610.88;

Streaming profile

Right context 13; VAD disabled; no word boosting

WER normalization

Lowercase and strip punctuation

Results qualify English test-clean only. They do not qualify multilingual or VAD-enabled accuracy.

3.2 Metric definitions#

  • Average WER: arithmetic mean of the word error rate calculated separately for each utterance. Every utterance contributes equally, regardless of duration.

  • Corpus WER: total substitutions, deletions, and insertions divided by the total number of reference words across the corpus. Longer utterances therefore contribute more than shorter utterances.

  • Perfect %: percentage of utterances with WER equal to zero.

  • High WER %: percentage of utterances with WER greater than or equal to 50%.

  • RTF (Real-Time Factor): processing time divided by audio duration. Values below 1 are faster than real time.

  • Speed vs real time: reciprocal of RTF (1 / RTF). For example, 50x means one second of processing handles approximately 50 seconds of audio.

  • TTFT (Time to First Token): average time from starting streaming inference until the first token is produced.

  • Finalization: average time required to process Stop and finalize the transcript.

  • Average chunk evaluation: average evaluation time for one streaming audio chunk.

3.3 Accuracy#

Word error rate measures transcription quality after the normalization described above. Lower WER is better; Perfect is the share of utterances transcribed without any word errors. Select a model tab to compare its four backend results.

Backend

Average WER %

Corpus WER %

Perfect %

High WER %

CUDA

2.220

1.873

76.374

0.344

D3D12

2.273

1.889

76.183

0.382

Vulkan

2.232

1.877

76.527

0.382

CPU

2.205

1.862

76.527

0.382

Backend

Average WER %

Corpus WER %

Perfect %

High WER %

CUDA

2.700

2.187

72.748

0.420

D3D12

2.706

2.178

72.824

0.420

Vulkan

2.743

2.208

72.634

0.496

CPU

2.703

2.191

72.672

0.382

Backend

Average WER %

Corpus WER %

Perfect %

High WER %

CUDA

3.038

2.625

69.389

0.458

D3D12

3.039

2.627

69.313

0.458

Vulkan

2.981

2.592

69.504

0.458

CPU

3.027

2.602

69.580

0.496

Backend

Average WER %

Corpus WER %

Perfect %

High WER %

CUDA

3.960

3.473

61.489

0.687

D3D12

3.979

3.467

61.832

0.687

Vulkan

3.955

3.469

61.718

0.649

CPU

3.989

3.492

61.603

0.763

3.4 Throughput#

RTF is processing time divided by audio duration; lower is better. Speed versus real time is 1 / RTF.

Backend

RTF

Speed vs real time

CUDA

0.004224

236.77x

D3D12

0.005976

167.34x

Vulkan

0.008742

114.39x

CPU

0.022384

44.68x

Backend

RTF

Speed vs real time

CUDA

0.003901

256.35x

D3D12

0.005872

170.30x

Vulkan

0.009037

110.66x

CPU

0.022794

43.87x

Backend

RTF

Speed vs real time

CUDA

0.013072

76.50x

D3D12

0.018899

52.91x

Vulkan

0.022567

44.31x

CPU

0.080289

12.46x

Backend

RTF

Speed vs real time

CUDA

0.013452

74.34x

D3D12

0.019700

50.76x

Vulkan

0.030377

32.92x

CPU

0.076572

13.06x

3.5 Streaming latency#

Streaming latency separates first-token responsiveness, per-chunk evaluation, and Stop finalization. Offline Parakeet models do not produce these streaming measurements.

Backend

TTFT ms

Finalization ms

Average chunk evaluation ms

CUDA

16.17

11.91

11.37

D3D12

24.51

15.99

16.77

Vulkan

26.74

21.29

19.56

CPU

93.65

80.81

68.25

Backend

TTFT ms

Finalization ms

Average chunk evaluation ms

CUDA

15.89

12.53

11.69

D3D12

24.20

17.39

17.34

Vulkan

34.50

30.74

26.45

CPU

85.70

76.93

65.10

Streaming timing measures benchmark execution, not simulated real-time microphone cadence. No separate warm-up result is reported.

4. Important Documentation in the Developer Pack#

Use this overview to select a model and deployment shape. Continue with the focused guide for the task you are implementing:

Goal

Document

Run a packaged sample immediately

Getting Started

Integrate the C++ API

Programming Guide

Use the CLI or 3D sample

Samples

Enable speech-state reporting

Silero VAD

Convert a supported NeMo ASR model to run with this plugin

NeMo ASR Model Conversion

5. Contents of the Developer Pack#

The pack combines ready-to-run binaries and models with sample source, public headers, build support, and model-conversion tools. Most integrations begin in bin/x64, data/nvigi.models, and include; the remaining folders support examples, rebuilding, debugging, or model preparation.

5.1 Top-level layout#

Each top-level folder has a distinct role:

Folder

Contents

bin/x64/

Runtime DLLs, samples, quantizer, and model verifier

data/

Model repositories and sample assets

docs/

Built developer documentation

include/

Public Nemotron ASR header

nvigi_core/

NVIGI Core headers, libraries, and runtime outputs

source/

Rebuildable sample source

symbols/

Debug symbols

tools/ and premake/

Model conversion and build support

5.2 Where to find things#

Use this table as a quick path from a development task to its location in the pack.

I want

Location

ASR plugin for a backend

bin/x64/nvigi.plugin.asr.nemotron-ggml.<backend>.dll

NVIGI framework

bin/x64/nvigi.core.framework.dll

Shipped models

data/nvigi.models/nvigi.plugin.asr.nemotron-ggml/{GUID}/

Public ASR header

include/nvigi_asr_nemotron.h

Command-line sample

bin/x64/nvigi.asr.sample.exe

Interactive sample

bin/x64/nvigi.3d.exe

Sample source

source/samples/

D3D12 runtime

bin/x64/D3D12/

Conversion tools

tools/nemo-quantization/

Debug symbols

symbols/

Keep the GUID directory intact when copying a model. All four backends read the same GGUF and adjacent configuration file.

6. Shipping Checklist#

Start with the common runtime and selected model, then add only the files required by the chosen backend and optional features. System and driver DLLs listed as external requirements come from Windows, the Microsoft runtime, or the GPU driver; they are not copied from the pack.

6.1 Always required#

Every deployment needs the NVIGI framework, one ASR backend, and at least one complete model GUID directory. Silero is needed only when VAD is enabled.

Item

Source

nvigi.core.framework.dll

bin/x64/

Selected Nemotron ASR backend DLL

bin/x64/

Selected model GGUF and adjacent config

Preserve its GUID directory under data/nvigi.models/nvigi.plugin.asr.nemotron-ggml/

Microsoft Visual C++ x64 runtime

Target system: MSVCP140.dll, VCRUNTIME140.dll, VCRUNTIME140_1.dll, VCOMP140.dll

Silero model directory

Only when streaming VAD is enabled

6.2 Per-backend requirements#

Select the tab for the backend being shipped. Preserve relative subdirectories such as bin/x64/D3D12.

Type

Requirement

Pack

nvigi.plugin.asr.nemotron-ggml.cpu.dll

External

No backend-specific driver

Type

Requirement

Pack

CUDA plugin, cudart64_12.dll, cublas64_12.dll, cublasLt64_12.dll

External

NVIDIA driver providing nvcuda.dll

Type

Requirement

Pack

nvigi.plugin.asr.nemotron-ggml.vk.dll

External

Vulkan loader/driver providing vulkan-1.dll

Type

Requirement

Pack

D3D12 plugin, dxcompiler.dll, dxil.dll, D3D12/D3D12Core.dll, D3D12/d3d12SDKLayers.dll

External

Windows d3d12.dll, dxgi.dll, and D3DCOMPILER_47.dll

Preserve the bin/x64/D3D12 subdirectory.

Add the HWI/CIG components only when using graphics interop or GPU scheduling. Samples, model tools, and PDBs are not required by an application runtime unless intentionally redistributed.

Appendix A. Full Directory Listing#

The abbreviated tree below identifies the customer-facing ASR files and their required relative locations. The pack also contains supporting build and sample assets.

bin/x64/
  nvigi.core.framework.dll
  nvigi.plugin.asr.nemotron-ggml.{cpu,cuda,d3d12,vk}.dll
  nvigi.asr.sample.exe
  nvigi.3d.exe
  asr_quantize.exe
  nvigi.asr.model_verify.exe
  D3D12/
data/nvigi.models/nvigi.plugin.asr.nemotron-ggml/
  {163BD6DC-248D-4238-BC9B-B76D2AB1C3DD}/
  {9739C15F-9A65-484D-B091-FFA4B9DDD0EE}/
  {9BC5E772-850B-4F34-AE71-311F39AB438D}/
  {36DFEC37-2FA0-4BE3-8D2F-03699FB433BE}/
  {8A5C059F-EDCA-49D1-875D-4F6CF208E5CE}/
include/nvigi_asr_nemotron.h
source/samples/
symbols/
tools/nemo-quantization/

Appendix B. Migration from Earlier Documentation#

This section is relevant only to integrations based on the earlier Riva-GGML naming. New integrations should use the current identifiers directly.

Previous identifier

Current identifier

nvigi.plugin.asr.riva-ggml.*

nvigi.plugin.asr.nemotron-ggml.*

nvigi_asr_riva.h

nvigi_asr_nemotron.h

nvigi::plugin::asr::riva_ggml

nvigi::plugin::asr::nemotron_ggml

ASRRivaGGML*

ASRNemotronGGML*

The previous identifiers are not aliases for current DLLs or public structures.