The Command-Line Basic Sample

The basic sample, nvigi.basic shows the basics of running a workflow of WAV (microphone) audio -> ASR (speech recognition) -> GPT (LLM). It allows direct typing “to” the LLM or “talking to” the LLM via a microphone and ASR.

These instructions make reference to a set of directories; the location of these directories differ between binary app developer pack and GitHub source. The documentation for each of these define the locations for these directories in the particular use:

<SDK_ROOT>: the root of the SDK Plugins tree, which contains the bin directory for the SDK.
<SDK_MODELS>: the root of the models tree for the SDK plugins, normally data/nvigi.models
<SDK_TEST>: the root of the test data tree for the SDK plugins, normally data/nvigi.test
<SDK_PLATFORM>: the target CPU architecture. See Platform Support for information on supported platforms.

Download Required Models

The basic sample requires the following models:

Plugin	Model Name	GUID
nvigi.plugin.asr.ggml.*	Whisper Small	5CAD3A03-1272-4D43-9F3D-655417526170
nvigi.plugin.gpt.ggml.*	Llama3.2 3b Instruct	01F43B70-CE23-42CA-9606-74E80C5ED0B6

See the top-level documentation that shipped with your development pack for information on how to download these models.

Cloud Models

Plugin	Model Name	GUID	URL
nvigi.plugin.gpt.cloud.rest	gpt-3.5-turbo	E9102ACB-8CD8-4345-BCBF-CCF6DC758E58	https://api.openai.com/v1/chat/completions
nvigi.plugin.gpt.cloud.rest	Llama 3.1 8B Instruct	40EE38B1-D89E-4B5C-A85C-B82FAB7541B9	https://integrate.api.nvidia.com/v1/chat/completions

NOTE: These are just two popular models, for more details on model repository please read the ProgrammingGuideAI located in the NVIGI Core SDK.

How to Use the Basic Sample

When run, the sample should launch a console (or use the one from which it was run); it will await user input. The user may do one of three things at the prompt:

Type a chat query as text and press enter. This will be passed directly to the LLM and the LLM response printed.
Type an exit command; “Q”, “q” or “quit”. This will cause the sample to exit.
Press enter with no text. This will start recording from the default Windows/DirectX recording device (e.g. a headset microphone). Pressing enter again will stop recording. Once the recording is complete, the audio will be sent to the ASR plugin and then the result of ASR printed to the console and passed to the LLM for response.

Run at Command Line

To run nvigi.basic from the command line, take the following steps (--models is a required argument):

Open a command prompt in <SDK_ROOT>
Run the command:

bin\<SDK_PLATFORM>\<Configuration>\nvigi.basic.exe --models <SDK_MODELS> --speaker <SDK_TEST>/nvigi.tts/chatterbox/spk_emb/aaron_turbo.json

In a standard layout binary development pack or GitHub source tree, launching from a current working directory of the <SDK_ROOT> directory, this is:

bin\<SDK_PLATFORM>\<Configuration>\nvigi.basic.exe --models data/nvigi.models --speaker data/nvigi.test/nvigi.tts/chatterbox/spk_emb/aaron_turbo.json

Here are the command line options:

Usage: nvigi.basic [options]

  -m, --models              model repo location (REQUIRED)
  --asr                     asr backend, 'cpu', 'cuda', or 'vulkan' (default: cuda)
  --asr-guid                asr model guid in registry format, in quotes (default: "{5CAD3A03-1272-4D43-9F3D-655417526170}")
  -a, --audio               audio file location (default: )
  --gpt                     gpt backend, 'cpu', 'cuda', 'vulkan', or 'cloud' - model GUID determines cloud endpoint (default: cuda)
  --gpt-guid                gpt model guid in registry format, in quotes (default: "{01F43B70-CE23-42CA-9606-74E80C5ED0B6}")
  -s, --sdk                 sdk location, (default: exe location)
  -t, --token               authorization token for the cloud provider (default: )
  --speaker                 path to the speaker embedding JSON of the voice you want to use (REQUIRED)
  --tts                     tts backend, 'cuda', 'vulkan', or 'd3d12'
  --tts-guid                tts model guid in registry format, in quotes (default: auto-selected based on backend)
  --vram                    the amount of vram to use in MB (default: 8192)

TTS Model Selection: If --tts-guid is not specified, the sample will automatically select the Chatterbox Turbo model based on the --tts backend:

GGML backends (--tts cuda, --tts vulkan, or --tts d3d12): Automatically uses {019BD494-0D97-7223-B9D5-C9286933B8B7} (Chatterbox Turbo). The speaker JSON passed via --speaker must match this model type.

If you specify a custom --tts-guid, ensure it is compatible with your chosen backend. Using mismatched backend/model pairs will result in errors.

Building the Sample

The sample is built as part of the SDK build process. For full build instructions, see Building and Debugging. After building, stage the binaries:

copy_sdk_binaries.bat [-x64|-arm64] [Release|Debug|Production]

This ensures all DLLs and the executable are in the same directory (bin\<SDK_PLATFORM>\<Configuration>, where <SDK_PLATFORM> is the target CPU architecture and is the selected build configuration; see Platform Support for information on supported platforms).

Run in Debugger

For the general procedure and background, see the Building and Debugging guide. The settings specific to this sample are:

Property	Value
VS Project	`nvigi/samples/nvigi.basic`
Command	`<SDK_ROOT>\bin\<SDK_PLATFORM>\<Configuration>\nvigi.basic.exe`
Command Arguments	`--models data\nvigi.models --speaker data\nvigi.test\nvigi.tts\chatterbox\spk_emb\aaron_turbo.json`
Working Directory	`<SDK_ROOT>\bin\<SDK_PLATFORM>\<Configuration>`