Is this page helpful?

RIVA ASR Developer Pack Overview#

1. What’s Included#

The RIVA ASR Plugin Pack provides automatic speech recognition (ASR) capabilities for NVIDIA In-Game Inferencing (NVIGI) applications. This pack includes:

  • Pre-built ASR plugin DLL (nvigi.plugin.asr.riva-ort)

  • Pre-configured Conformer-CTC model with full language support

  • Command-line and 3D sample applications

  • Complete source code for samples (rebuild from source)

  • Comprehensive documentation and programming guides

2. Automatic Speech Recognition Models#

2.1 Included Models#

The pack includes the English Conformer-CTC model bundled with all necessary components. Parakeet-0.6B and multilingual models are available via download.

Plugin

Model Name

GUID

Language

Status

nvigi.plugin.asr.riva-ort

Conformer-CTC

B5D4F28E-3DB9-4FDA-A78B-A569AA22FCD0

en-US

Bundled

nvigi.plugin.asr.riva-ort

Parakeet-0.6B

53211BAC-8B34-4AE5-83C3-9B4CA0B2AE6D

en-US

Download

nvigi.plugin.asr.riva-ort

Conformer-CTC-French

7B9ED4D0-D15C-450F-BCC8-8FA7B460BFC0

fr-FR

Download

nvigi.plugin.asr.riva-ort

Conformer-CTC-German

632AA9FC-5C76-460D-B915-791E41F40028

de-DE

Download

nvigi.plugin.asr.riva-ort

Conformer-CTC-Italian

33A53EA0-AD25-4CF1-B4FB-9B1513FDA24F

it-IT

Download

nvigi.plugin.asr.riva-ort

Conformer-CTC-Spanish

20AD4848-5CB4-49CA-B48F-AF1AA61B4EB2

es-ES

Download

nvigi.plugin.asr.riva-ort

Conformer-CTC-Korean

14BA55B2-0357-4B7B-B021-B9F1D8DFB77A

ko-KR

Download

nvigi.plugin.asr.riva-ort

Conformer-CTC-Japanese

29A8FAFA-A9BB-485C-AEEF-BFF4512DFE91

ja-JP

Download

nvigi.plugin.asr.riva-ort

Conformer-CTC-Mandarin

0DDAD855-F9C3-42B4-AEED-18C9D0440923

zh-CN

Download

Note

To download Parakeet-0.6B or multilingual models, run download_data.bat from the pack root directory.

To download all available models: download_data.bat

To download specific models only: download_data.bat parakeet french mandarin

Available model names: parakeet, french, german, italian, spanish, korean, japanese, mandarin

2.2 Model Components#

The Conformer-CTC model includes:

  • Acoustic Model (acoustic_mdl.onnx) - Main speech recognition model

  • Preprocessing Model (preproc_mdl.onnx) - Audio feature extraction

  • Punctuation Model (punc_mdl.onnx) - Automatic punctuation and capitalization

  • Language Model (en-US.bin) - KenLM 4-gram model for beam search decoding

  • Vocabularies & Lexicon - Token mappings, phonetic dictionaries, and label IDs

2.3 Model Capabilities#

  • Multilingual: Support for 8 languages – English, French, German, Italian, Spanish, Korean, Japanese, and Mandarin Chinese

  • Streaming & Offline: Real-time streaming recognition or batch offline transcription

  • Decoding Strategies: Greedy or beam search with language model

  • Punctuation: Optional automatic punctuation and capitalization (English models)

  • Word Boosting: Custom vocabulary emphasis (positive/negative scores)

  • Silence Detection: Automatic speech/pause detection in streaming mode

  • Multi-backend: CPU (ONNX Runtime), DirectML (GPU), or TensorRT-RTX (NVIDIA GPUs)

3. Important Documentation in the Developer Pack#

The main documentation for using the pack includes:

  • Samples

    • The best “starting” doc, as the sample is provided pre-compiled and ready-to-run. It describes how to run, recompile, and debug the samples.

  • RIVA ASR Plugin Programming Guide

    • Detailed documentation on how to program for the RIVA ASR plugin, including streaming and offline speech recognition, model loading, and API usage

4. Samples Included#

The pack includes pre-built sample applications with full source code:

4.1 Command-Line Samples#

  • nvigi.asr.sample - Real-time streaming ASR demo

    • Microphone input or WAV file processing

    • Live partial and final transcriptions

    • Configurable chunk size, context windows, and decoding strategies

4.2 3D GUI Sample#

  • nvigi.3d - Interactive 3D application with ASR integration

    • Voice-controlled interface using speech recognition

    • Visual feedback and transcription display

    • Demonstrates real-world ASR integration in a 3D application

5. Contents of the Developer Pack#

The pack is organized as follows:

riva_asr_pack/
├── bin/x64/                                # Pre-built binaries
│   ├── nvigi.plugin.asr.riva-ort.gpu.dll      # ASR plugin DLL
│   ├── nvigi.plugin.asr.riva-ort.cpu.dll      # ASR plugin DLL
│   ├── nvigi.asr.sample.exe            # Streaming ASR sample
│   └── nvigi.3d.exe                       # 3D GUI sample
│
├── include/                                # NVIGI SDK headers
│   └── nvigi/                             # Plugin API headers
│
├── source/                                 # Source code
│   ├── samples/                           # Sample source code (rebuildable)
│   │   ├── nvigi.asr.sample/
│   │   └── nvigi.3d/
│   └── plugins/nvigi.asr/                 # Plugin source (reference only)
│
├── data/
│   ├── nvigi.models/                      # ASR models
│   │   └── nvigi.plugin.asr.riva-ort/
│   │       ├── {B5D4F28E-...}/            # Conformer-CTC (en-US)
│   │       ├── {53211BAC-...}/            # Parakeet-0.6B (en-US)
│   │       ├── {7B9ED4D0-...}/            # Conformer-CTC-French (fr-FR)
│   │       ├── {632AA9FC-...}/            # Conformer-CTC-German (de-DE)
│   │       ├── {33A53EA0-...}/            # Conformer-CTC-Italian (it-IT)
│   │       ├── {20AD4848-...}/            # Conformer-CTC-Spanish (es-ES)
│   │       ├── {14BA55B2-...}/            # Conformer-CTC-Korean (ko-KR)
│   │       ├── {29A8FAFA-...}/            # Conformer-CTC-Japanese (ja-JP)
│   │       └── {0DDAD855-...}/            # Conformer-CTC-Mandarin (zh-CN)
│   │           Each model directory contains:
│   │           ├── acoustic_mdl.onnx      # Main speech recognition model
│   │           ├── preproc_mdl.onnx       # Audio feature extraction
│   │           ├── punc_mdl.onnx          # Punctuation (English models only)
│   │           ├── *.bin                  # KenLM language model
│   │           └── [vocabularies and lexicons]
│   │
│   └── nvigi.test/                        # Test data
│       └── nvigi.asr/                     # WAV files for testing
│           ├── jfk.wav
│           ├── micro-machines.wav
│           ├── fugatto.wav
│           └── whisper-stream-test.wav
│
└── docs/                                   # Documentation
    ├── samples.md                                      # Sample usage and rebuild guide
    ├── programming-guide.md      # Complete API reference
    └── getting-started.md                              # Setup instructions

6. Runtime Dependencies#

This section describes the DLLs required to run the ASR plugin and sample applications. Game developers integrating ASR should focus on Plugin Dependencies.

6.1 Plugin Dependencies#

These DLLs are required when integrating the NVIGI ASR plugin into your game or application.

NVIGI Core Framework#

DLL

Size

Purpose

nvigi.core.framework.dll

0.25 MB

Core NVIGI plugin framework - loads and manages plugins

nvigi.plugin.hwi.common.dll

0.04 MB

Common hardware interface - shared GPU utilities

nvigi.plugin.hwi.cuda.dll

0.07 MB

CUDA hardware interface - required for TensorRT-RTX backend

nvigi.plugin.hwi.d3d12.dll

0.09 MB

D3D12 hardware interface - required for DirectML backend and D3D12 interop

amd_ags_x64.dll

0.16 MB

AMD GPU Services - cross-vendor GPU support

ASR Plugin (choose one or both)#

DLL

Size

Purpose

nvigi.plugin.asr.riva-ort.gpu.dll

0.57 MB

GPU-accelerated ASR (TensorRT-RTX or DirectML backends)

nvigi.plugin.asr.riva-ort.cpu.dll

0.54 MB

CPU-only ASR (ONNX Runtime CPU backend)

ONNX Runtime (required for GPU plugin)#

DLL

Size

Purpose

Dependency Of

onnxruntime.dll

20.27 MB

ONNX Runtime inference engine

Core dependency

onnxruntime_providers_shared.dll

0.01 MB

Shared execution provider infrastructure

onnxruntime.dll

DirectML.dll

17.67 MB

DirectML execution provider (GPU inference on any D3D12 GPU)

onnxruntime.dll

onnxruntime_providers_nv_tensorrt_rtx.dll

0.54 MB

TensorRT-RTX execution provider (optimized for NVIDIA RTX GPUs)

onnxruntime.dll

tensorrt_rtx_1_2.dll

153.13 MB

TensorRT-RTX runtime engine

onnxruntime_providers_nv_tensorrt_rtx.dll

tensorrt_onnxparser_rtx_1_2.dll

2.87 MB

ONNX model parser for TensorRT-RTX

tensorrt_rtx_1_2.dll

CUDA / CIG Scheduler#

DLL

Size

Purpose

cudart64_12.dll

0.55 MB

CUDA 12 runtime - required for TensorRT-RTX backend

cupti64_2025.1.0.dll

4.27 MB

CUDA Profiling Tools Interface - used by CIG scheduler

cig_scheduler_settings.dll

0.04 MB

CUDA-in-Graphics scheduler - balances GPU between graphics and inference

System Dependencies (not shipped - expected on target system)#

DLL

Source

Required For

d3d12.dll

Windows 10+

DirectML backend, D3D12 interop

dxgi.dll

Windows 10+

GPU enumeration, device creation

nvcuda.dll

NVIDIA Driver

TensorRT-RTX backend, CUDA operations

6.2 Application Dependencies#

These additional DLLs are required only for the sample applications. All Plugin Dependencies above also apply.

3D Sample (nvigi.3d.exe)#

DLL

Size

Location

Purpose

D3D12Core.dll

3.20 MB

bin/x64/D3D12/

D3D12 Agility SDK - provides consistent D3D12 features

d3d12SDKLayers.dll

4.84 MB

bin/x64/D3D12/

D3D12 validation/debug layers

6.3 Dependency Hierarchy#

ASR Plugin (GPU)
├── NVIGI Core Framework
│   ├── nvigi.core.framework.dll
│   ├── nvigi.plugin.hwi.common.dll
│   ├── nvigi.plugin.hwi.cuda.dll ──→ [nvcuda.dll, cudart64_12.dll]
│   └── nvigi.plugin.hwi.d3d12.dll ──→ [d3d12.dll, dxgi.dll]
│
├── ONNX Runtime
│   ├── onnxruntime.dll
│   ├── onnxruntime_providers_shared.dll
│   ├── DirectML.dll ──→ [d3d12.dll]
│   └── TensorRT-RTX Provider
│       ├── onnxruntime_providers_nv_tensorrt_rtx.dll
│       ├── tensorrt_rtx_1_2.dll ──→ [nvcuda.dll]
│       └── tensorrt_onnxparser_rtx_1_2.dll
│
├── Language Model
│   ├── flashlight-text.dll
│   ├── flashlight-text-kenlm.dll
│   └── kenlm.dll
│
└── CIG Scheduler (optional, for graphics+inference balancing)
    ├── cig_scheduler_settings.dll
    └── cupti64_2025.1.0.dll