NVIGI - Programming Guide

This guide primarily focuses on the general use of the In-Game Inferencing, including how to initialize it and utilize its interfaces.

IMPORTANT: This guide might contain pseudo code, for the up to date implementation and source code which can be copy pasted please see the SDK Samples

Table of Contents

Introduction
Key Concepts
Typed And Versioned Structures
Thread Safety
Security
Core API
Validation
D3D Considerations
- Streamline
- Agility SDK
3rd Party Dependencies
Troubleshooting
- Crash Dumps and Exceptions

INTRODUCTION

NVIGI functions as a plugin manager that provides secure plugin loading and an explicit API, giving the host application complete control over every aspect of NVIGI’s usage.

Key Concepts

Typed and versioned structures are used for all data and C-style interfaces.
- Structures can be chained together as needed
- Structures are ABI-compatible and backwards-compatible
- Structures can contain C-style APIs (aka interfaces/function-pointers) or just plain data (aka parameters or properties)
Each plugin implements at least one C-style interface which is provided to the host application and other plugins to use
Interfaces are completely custom and it is up to the plugins to decide what functionality is needed in them
Different plugins can implement identical interfaces if needed, normally they would use different “backends” to provide same functionality
Plugins are developed independently and can be updated independently (as needed) in any application (even after it has shipped)
- Each plugin defines the minimum specification required for it to run (minimum driver, OS versions etc.)
Plugins have unique identifiers in the following namespace format nvigi::plugin::$name{::$backend::$api}::kId where backend and API are optional
The core component nvigi.core.framework enumerates all available plugins, determines which plugins can run on user’s system and what interfaces do they export
- Not all plugin locations need to be known when nvigi.core.framework is initialized
- However, all shared dependencies must be known and stored in a single location

Typed And Versioned Structures

Typed and versioned structures can be used to provide input data, obtain output results and call custom C-style APIs.

Input Data

In this case, the host application provides information to NVIGI. Depending on scenario, multiple inputs (structures) can be chained together and later sought for with the findStruct API. Here are some examples:

//! We are creating local GPT instance while providing common and D3D12 related information

//! Common
nvigi::CommonCreationParameters common{};
common.numThreads = myNumCPUThreads; // How many CPU threads is instance allowed to use 
common.vramBudgetMB = myVRAMBudget;  // How much VRAM is instance allowed to occupy
common.utf8PathToModels = myPathToNVIGIModelRepository; // Path to provided NVIGI model repository (using UTF-8 encoding)
common.modelGUID = "{175C5C5D-E978-41AF-8F11-880D0517C524}"; // Model GUID, for details please see NVIGI models repository

//! GPT specific 
nvigi::GPTCreationParameters params{};
params.maxNumTokensToPredict = 200;
params.contextSize = 512;
params.seed = -1;

Note that we are chaining common properties to our main properties like this:

    if(NVIGI_FAILED(params.chain(common)))
    {
        // Make sure NOT to share structures across different chains unless you ensure they are added at the end of each chain
    }

IMPORTANT: Chaining can fail when sharing the same structure across multiple chains which is NOT allowed unless the shared structure is the leaf (at the end in all chains; this is NOT recommended practice)

Now we provide additional information about D3D12 context, assuming the host application is using D3D graphics API:

//! D3D12 specific 
nvigi::D3D12Parameters d3d12Params{};
d3d12Params.device = myDevice;
d3d12Params.queue = myQueue;

Then we chain it together like this:

    if(NVIGI_FAILED(params.chain(d3d12Parameters)))
    {
        // Handle error
    }

Finally, we end up with the following chained input structure which can be provided to NVIGI interface(s) for processing:

nvigi::GPTCreationParameters -> nvigi::D3D12Parameters -> nvigi::CommonCreationParameters

NVIGI plugins, in this case GPT, will use findStruct API to find all mandatory and optional inputs which are chained together.

Output Data

When calling various NVIGI APIs it is a common case that they provide specific output(s) as typed and versioned structures. Here is one example:

//! Obtain interface, note that interface is just a special typed and version structure
nvigi::IGeneralPurposeTransformer* igpt{};
nvigiGetInterface(nvigi::plugin::gpt::ggml::cuda::kId, &igpt, params.nvigiLoadInterface);

//! Tell our GPT interface where to find models
nvigi::GPTCreationParameters params{};
common.utf8PathToModels = myPathToModelRepo;
//! Chain it together so GPT interface can find it
if(NVIGI_FAILED(params.chain(common)))
{
    // Handle error
}

//! Now we obtain capabilities and requirements for this instance, again as typed and versioned structure but containing just data
nvigi::CommonCapabilitiesAndRequirements* caps{};
getCapsAndRequirements(igpt, params, &caps);

By design, NVIGI will chain together any additional output(s) so the requester can use the findStruct function to check if they are present like this:

//! Check if optional output is chained with caps
auto sampler = findStruct<nvigi::GPTSamplerParameters>(*caps);
if (sampler)
{
    //! This interface supports sampler parameters
    //!
    //! They can be modified as needed and chained to the nvigi::GPTCreationParameters when creating an instance
    sampler->penalizeNewLine = true;
    if(NVIGI_FAILED(params.chain(sampler)))
    {
        // Handle error
    }
}

NOTE: In this example, since interface in question supports nvigi::GPTSamplerParameters, sampler parameters can be chained together with the nvigi::GPTCreationParameters and serve as input (see the above section) if it is required to change any sampler related options when creating the instance.

Thread Safety

By default, any API in NVIGI is NOT thread safe unless stated otherwise in the comment above the API declaration. For example:

//! ....
//!
//! This method is NOT thread safe.
nvigi::Result(*createInstance)(const nvigi::NVIGIParameter* params, nvigi::InferenceInstance** instance);

Note that the line immediately above the declaration explicitly states that this method is NOT thread safe. It is the host’s responsibility to provide synchronization in such scenarios.

Security

Digital Signatures

When running on Windows and in production configuration, NVIGI core framework will enforce digital signature check on all NVIGI plugin libraries but NOT on their dependencies. For example:

nvigi.plugin.$name.$backend.$api.dll  # signed by NVDA, signature checked by `nvigi.core.framework.dll`
├── cudaRt_12.dll                     # signed by NVDA, signature NOT checked by `nvigi.core.framework.dll`
├── tensorRT_10.dll                   # signed by NVDA, signature NOT checked by `nvigi.core.framework.dll`
├── libprotobuf.dll                   # not signed, up to the host app to enforce security
├── zlib.dll                          # not signed, up to the host app to enforce security
└── kernel32.dll                      # system library, ignored since it is signed by the OS and in the secure location

To prevent hackers from replacing nvigi.core.framework.dll with potentially malicious code it is expected that the host application uses API located in nvigi_security.h to ensure that digital signature is valid before loading any NVIGI libraries. Here is an example:

//! IMPORTANT: Always use absolute path to DLL
auto pathToNVIGICore = std::filesystem::path(myPathToNVIGISDK) / L"nvigi.core.framework.dll";
if(nvigi::security::verifyEmbeddedSignature(pathToNVIGICore.wstring().c_str()))
{
  //! Safe to load core and initialize NVIGI

  //! IMPORTANT: Always use absolute path to DLL
  HMODULE lib = LoadLibraryW(pathToNVIGICore.wstring().c_str());
}

IMPORTANT: Failure to check for digital signature on nvigi.core.framework.dll can result in executing potentially dangerous code with a wide range of consequences.

As mentioned above, nvigi.core.framework.dll will NOT check any signatures on 3rd party dependencies. This is a security vulnerability and must be addressed by the host application either by enforcing CRC checks on all DLLs or some other method which ensures that libraries haven’t been tampered with before loading them.

3rd Party Shared Libraries

When running on non NVIDIA hardware shared libraries like amd_ags_x64.dll can be loaded by nvigi.core.framework.dll to check hardware capabilities, driver version etc. If these 3rd party libraries are NOT digitally signed NVIGI will NOT use secure loading procedures so failure to ensure their integrity, by the host app, can result in executing potentially dangerous code with a wide range of consequences.

Elevated Privileges

NOTE: This is related only to Windows platform, Linux users can skip this section

If the host process is running with elevated privileges NVIGI core will attempt to downgrade some of them to mitigate security risks. If this behavior is not desirable it can be overridden by setting PreferenceFlags::eDisablePrivilegeDowngrade flag.

IMPORTANT: Please note that when opting out from the privilege downgrade, the host application is taking over the responsibility for any security breaches that might occur in this scenario, it is highly recommended to install the NVIGI bits in secure location which can be modified only by admin users.

Here is the list of privileges in question:

SE_LOAD_DRIVER_NAME
SE_DEBUG_NAME
SE_TCB_NAME
SE_ASSIGNPRIMARYTOKEN_NAME
SE_SHUTDOWN_NAME
SE_BACKUP_NAME
SE_RESTORE_NAME
SE_TAKE_OWNERSHIP_NAME
SE_IMPERSONATE_NAME

Core API

The core API is rather minimalistic and located in the nvigi.h header.

VERY IMPORTANT: Always make sure to securely load nvigi.core.framework.dll before calling any NVIGI APIs (see the security section above)

Initialization

//! Initializes the NVIGI framework
//!
//! Call this method when your application is initializing
//!
//! NOTE: On Windows, host is NOT supposed to change DLL search paths in any way while this method is running without proper synchronization.
//!
//! @param pref Specifies preferred behavior for the NVIGI framework (NVIGI will keep a copy)
//! @param pluginInfo Optional pointer to data structure containing information about plugins, user system
//! @param sdkVersion Current SDK version
//! @returns nvigi::kResultOk if successful, error code otherwise (see nvigi_result.h for details)
//!
//! This method is NOT thread safe and will temporarily change DLL search path on Windows.
NVIGI_API nvigi::Result nvigiInit(const nvigi::Preferences &pref, nvigi::PluginAndSystemInformation** pluginInfo = nullptr, uint64_t sdkVersion = nvigi::kSDKVersion);

This function initializes NVIGI SDK based on the provided preferences. It enumerates available plugins and, if requested, can report back detailed information on what plugins are available, which are compatible with the user’s system and which are not, what are the basic requirements for each plugin etc. Here is an example:

nvigi::Preferences pref{};
pref.logLevel = nvigi::LogLevel::eDefault; // eVerbose for more detailed log
pref.showConsole = true; // false when shipping
pref.numPathsToPlugins = count(myPathsToNVIGIPlugins);
//! NOTE: Plugins and their custom dependencies can be split up across different locations. Shared dependencies must be in one location (see below)
pref.utf8PathsToPlugins = myPathsToNVIGIPlugins;
//! NOTE: Location for shared dependencies, if not provided NVIGI will assume that there are NO shared dependencies only custom ones and they are all next to their respective plugins.
pref.utf8PathToDependencies = myPathToNVIGISharedPluginDependencies;
pref.logMessageCallback = myCallback; // OPTIONAL
pref.utf8PathToLogsAndData = myPathToLogs; // OPTIONAL but highly recommended, much easier to track down any issues

NOTE: In non-production builds preferences can be overridden via nvigi.core.framework.json file if placed next to the nvigi.core.framework.dll. This allows user to change logging level, path to plugins, log file etc. without having to change the code.

// Optional info about system and plugins
nvigi::PluginAndSystemInformation* info{};
if(NVIGI_FAILED(result, nvigiInit(pref, &info, nvigi::kSDKVersion)))
{
    // Handle error
}

Note that not all plugins need to be enumerated here, later on when requesting interfaces it is possible to provide additional paths to search. In fact, it is completely valid to initialize IGI without any plugin directories specified:

pref.numPathsToPlugins = 0;
pref.utf8PathsToPlugins = nullptr;

However, in this scenario where no plugin directories are provided, it is mandatory to provide utf8PathToPlugins when calling nvigiLoadInterface (see below for more details). The main use of explicit plugin directory loading via passing no paths to nvigiInit and passing explicit path to nvigiLoadInterface is to allow the host application to select different, possibly conflicting plugins much later in the application flow. Different plugin directories can include different versions of the same plugin if need be. As noted, the sect of directories passed to nvigiInit may not include any pairs/sets of conflicting plugins. But a later call to nvigiLoadInterface can include one of the paths that contains a potentially conflicting plugin.

Note that providing no plugin directories in init and providing a plugin directory when loading an interface may not be faster than providing the plugin directories to nvigiInit, as NVIGI will scan all of the plugins in the directory given to nvigiLoadInterface to find the plugin that implements the requested interface. Loading several plugins from the same explicit directory via nvigi[Get,Load]Interface can lead to most of the entire directory being scanned multiple times, which can be inefficient. Delayed plugin directory specification is not specifically designed for nvigiInit optimization.

Initial scanning of the shared plugin directory in nvigiInit can be optimized by not placing unused plugins in the directory.

IMPORTANT: NVIGI SDK can come in various configurations (debug, release, production etc.) which can contain different 3rd party dependencies. To avoid runtime issues it is absolutely essential to use correct set of NVIGI plugins combined with matching set of 3rd party dependencies - do NOT mix debug, release etc.

Shutdown

Once NVIGI is no longer needed the following API needs to be called to release any used resources and unload any plugins which might be resident in memory:

//! Shuts down the NVIGI module
//!
//! Call this method when your application is shutting down. 
//!
//! @returns nvigi::kResultOk if successful, error code otherwise (see nvigi_result.h for details)
//!
//! This method is NOT thread safe.
NVIGI_API nvigi::Result nvigiShutdown();

Interfaces

As mentioned in the introduction, each plugin is assigned a unique identifier and it must implement at least one interface which is represented as a typed and versioned structure. Host application requests an interface which is needed to perform specific tasks and releases it once it is no longer needed. Here is the API which loads an interface:

//! Loads an interface for a specific NVIGI feature
//!
//! Call this method when specific interface is needed.
//!
//! NOTE: Interfaces are reference counted so they all must be released before underlying plugin is released.
//!
//! @param feature Specifies feature which needs to provide the requested interface
//! @param interfaceType Type of the interface to obtain
//! @param interfaceVersion Minimal version of the interface to obtain
//! @param interface Pointer to the interface
//! @param utf8PathToPlugin Optional path to a new plugin which provides this interface
//! @returns nvigi::kResultOk if successful, error code otherwise (see nvigi_result.h for details)
//! 
//! This method is NOT thread safe.
NVIGI_API nvigi::Result nvigiLoadInterface(nvigi::PluginID feature, const nvigi::UID& interfaceType, uint32_t interfaceVersion, void** _interface, const char* utf8PathToPlugin);

It is important to emphasize that this API allows the host to provide an additional path to a plugin which implements this interface. In other words, not all plugin locations need to be known when nvigiInit is called.

NOTE: This API will load and make resident in memory the shared library which matches the PluginID (unless it is already loaded due to earlier request for an interface it implements)

It is highly recommended to use the templated helpers nvigiGetInterface or nvigiGetInterfaceDynamic depending on the linking option used in the host application:

//! Helper method when statically linking NVIGI framework
//! 
template<typename T>
inline nvigi::Result nvigiGetInterface(nvigi::PluginID feature, T** _interface)

//! Helper method when dynamically loading NVIGI framework
//! 
template<typename T>
inline nvigi::Result nvigiGetInterfaceDynamic(nvigi::PluginID feature, T** _interface, PFun_nvigiLoadInterface* func)

For example:

nvigi::IGeneralPurposeTransformer* igpt{};

// Static linking, as an example requesting interface from ggml cuda backend
if(NVIGI_FAILED(result, nvigiGetInterface(nvigi::plugin::gpt::ggml::cuda::kId, &igpt)))
{
    // Handle error
}
 
// Dynamic `nvigi.core.framework` loading, again as an example requesting interface from ggml cuda backend
if(NVIGI_FAILED(result, nvigiGetInterfaceDynamic(nvigi::plugin::gpt::ggml::cuda::kId, &igpt, nvigiLoadInterfaceFunction)))
{
    // Handle error
}

When a certain interface is no longer needed, it should be unloaded so that the underlying shared library which implements it can be released. This is achieved with the following API:

//! Unloads an interface for a specific NVIGI feature
//!
//! Call this method when specific interface is no longer needed
//!
//! @param feature Specifies feature which provided the interface
//! @param interface Pointer to the interface
//! @returns IResult interface with code nvigi::kResultOk if successful, error code otherwise (see nvigi_result.h for details)
//!
//! This method is NOT thread safe.
NVIGI_API nvigi::Result nvigiUnloadInterface(nvigi::PluginID feature, void* _interface);

NOTE: Interfaces are reference counted so the underlying shared library will be released only when ALL references to the requested interface(s) are released.

Validation

Once successfully initialized the optional nvigi::PluginAndSystemInformation, if provided as shown in the above section when calling nvigiInit, contains useful information which can be used to determine if specific plugin and or interface is available. NVIGI comes with various helpers which can be used as shown below:

Check If Specific Plugin Is Supported

//! Replace $name::$backend::$api as appropriate
if(NVIGI_FAILED(result, nvigi::getPluginStatus(info, nvigi::plugin::$name::$backend::$api::kId)))
{
  // Not supported, let's find out why
  if(result == nvigi::kResultItemNotFound)
  {
    // Plugin is missing from the search paths provided in preferences
  }
  else if(result == nvigi::kResultMissingDynamicLibraryDependency)
  {
    // Check logs, shared libraries either missing or misplaced
  }
  else if(result == nvigi::kResultOSOutOfDate)
  {
    nvigi::Version version;
    nvigi::getPluginRequiredOSVersion(info, nvigi::plugin::$name::$backend::$api::kId, version);
  }
  else if(result == nvigi::kResultDriverOutOfDate)
  {
    nvigi::Version version;
    nvigi::getPluginRequiredAdapterDriverVersion(info, nvigi::plugin::$name::$backend::$api::kId, version);
  }
  else if(result == nvigi::kResultNoSupportedHardwareFound)
  {
    nvigi::VendorId vendor;
    nvigi::getPluginRequiredAdapterVendor(info, nvigi::plugin::$name::$backend::$api::kId, vendor);
    uint32_t arch; // 0 indicates any architecture is fine
    nvigi::getPluginRequiredAdapterArchitecture(info, nvigi::plugin::$name::$backend::$api::kId, arch);
  }
}

Check If Specific Plugin Exports An Interface

//! Replace $name::$backend::$api and $some_interface_name as appropriate
if(NVIGI_FAILED(result, nvigi::isPluginExportingInterface(info, nvigi::plugin::$name::$backend::$api::kId, nvigi::$some_interface_name)))
{
  // Not supported, check the result
}

IMPORTANT: At this point NO SHARED LIBRARIES (PLUGINS) ARE LOADED NOR RESIDENT IN MEMORY, only upon an explicit request for an interface implemented by a specific plugin will that plugin be loaded.

D3D Considerations

Streamline

Care should be taken when integrating NVIGI into an existing application that is also using a D3D object wrapper like Streamline. The queue/device parameters passed to NVIGI must be the native objects, not the app-level wrappers. In the case of Streamline, this means using slGetNativeInterface to retrieve the base interface object before passing it to NVIGI.

Microsoft Agility SDK

Please note that D3D12 based plugins require a device which is at least SM 6.6 capable, this feature might not be available on all Windows versions therefore it must be enabled via the Agility SDK. The additional benefit of including the latest Agility SDK is the performance enhancement which comes with the introduction of the new heap type D3D12_HEAP_TYPE_GPU_UPLOAD. This new feature enables simultaneous CPU and GPU access to VRAM via the Resizable BAR (ReBAR) mechanism-was introduced to the DirectX 12 API through the Direct3D Agility SDK and corresponding Windows updates. This feature allows for more efficient data transfers, reducing the need for CPU-to-GPU copy operations and potentially improving performance in certain scenarios. For more details please visit the MS Agility SDK blog

Feature	First Supported Windows OS	First Supported Agility SDK Version
GPU UPLOAD HEAP (ReBAR)	Windows 11 Insider Preview Build 26080 or later	1.613.0

IMPORTANT: Please note that on some systems ReBAR must be explicitly enabled in the BIOS.

3rd Party Dependencies

EXTREMELY IMPORTANT PLEASE READ: If the host application is using the exact same dependency as NVIGI but different version this can lead to runtime issues and random crashes. Please contact NVIDIA to discuss possible solutions.

NVIGI SDK can come in various configurations (debug, release, production) which can contain different 3rd party dependencies.

To avoid runtime issues please make sure that:

NVIGI shared dependencies (CUDA, tensorRT etc.) are stored in one location pointed by the nvigi::Preferences::utf8PathToDependencies parameter.
Correct set of NVIGI plugins is combined with matching set of 3rd party dependencies (debug and release dependencies are NOT mixed up etc.)
nvigi::Preferences::utf8PathsToPlugins and nvigi::Preferences::utf8PathToDependencies always point to the appropriate locations based on the build configuration.
On Windows, AddDLLDirectory or similar APIs are NOT used to manage NVIGI paths implicitly - this prevents NVIGI plugins from obtaining the location of their dependencies.
On Linux, LD_LIBRARY_PATH includes path(s) to NVIGI dependencies before host application starts

In addition to the above, please note the following:

nvigiInit and nvigiLoadInterface are NOT thread safe and they will TEMPORARILY modify DLL search path based on the information provided by the host.
- Host application should NOT do any DLL related operations while the above mentioned APIs are running without an explicit synchronization
If nvigi::Preferences::utf8PathToDependencies is NOT provided NVIGI will assume that there are NO shared dependencies and any plugin specific dependencies all MUST be next to their respective plugin(s).
Individual plugins might need the nvigi::Preferences::utf8PathToDependencies to dynamically load their own dependencies and validate digital signatures on them
If plugin dependency is NOT located in nvigi::Preferences::utf8PathsToPlugins or nvigi::Preferences::utf8PathToDependencies that plugin will fail to load to avoid DLL ABI issues.

Runtime Scenarios With Dependencies

NVIGI and host application do NOT share any dependencies or share a subset of identical dependencies - same name(s), compatible ABI(s)
- No runtime issues but keep in mind that NVIGI expects to load its dependencies from nvigi::Preferences::utf8PathsToPlugins or nvigi::Preferences::utf8PathToDependencies
NVIGI and host application share a subset of dependencies with the same name(s) but incompatible ABI(s)
- Unless each API exported by the dependencies in question is accessed explicitly using HMODULE and GetProcAddress there can be random crashes and instability
- Preferably, the host application should rename the dependencies in question to avoid ABI collision(s)
- Alternatively, either NVIGI plugin(s) or the host app should be recompiled to use the same version of shared dependencies

Plugin And Dependencies Deployment Examples

Everything In One Location:

// This is a very common and valid way to distribute NVIGI with your application
$some_path/nvigi
│
├──nvigi.core.framework.dll
├──nvigi.plugin.gpt.ggml.cuda.dll
├──nvigi.plugin.asr.ggml.cuda.dll
├──cudart64_12.dll
├──ggml_40f25557_cuda.dll
└──zlib1.dll

NOTE: In this scenario both nvigi::Preferences::utf8PathsToPlugins and nvigi::Preferences::utf8PathToDependencies point to the same location - $some_path/nvigi

Split In Multiple Locations

Core and shared dependencies together

// Care must be taken not to duplicate plugins or dependencies in this setup
$some_path/nvigi/core
│
├──nvigi.core.framework.dll
└──cudart64_12.dll // Shared by many plugins so must be in one location

NOTE: Here nvigi::Preferences::utf8PathToDependencies points to $some_path/nvigi/core but nvigi::Preferences::utf8PathsToPlugins points to different locations (see below)

GPT plugin(s)

// Care must be taken not to duplicate plugins or dependencies in this setup
$some_path/nvigi/gpt
│
├──nvigi.plugin.gpt.ggml.cuda.dll
└──tensorRT_10.dll // This is OK as long as other plugins don't use it, this is version 10

ASR plugin(s)

// Care must be taken not to duplicate plugins or dependencies in this setup
$some_path/nvigi/asr
│
├──nvigi.plugin.asr.ggml.cuda.dll
└──tensorRT_11.dll // This is OK as long as other plugins don't use it, this is version 11

NOTE: In the above example, if a new plugin is introduced and uses tensorRT_10.dll for example, this shared library now must move to $some_path/nvigi/core otherwise we have invalid configuration with duplicated dependency.

Invalid Configurations

IMPORTANT: Note that in this scenario shared library is duplicated across different locations which is NOT allowed.

$some_path/nvigi/core
│
├──nvigi.core.framework.dll
└──cudart64_12.dll

GPT plugin(s)

$some_path/nvigi/gpt
│
├──nvigi.plugin.gpt.ggml.cuda.dll
└──cudart64_12.dll // !!! duplicates NOT allowed

ASR plugin(s)

$some_path/nvigi/asr
│
├──nvigi.plugin.asr.ggml.cuda.dll
└──cudart64_12.dll // !!! duplicates NOT allowed

Collisions With The Host

In this example, host uses AddDllDirectory to include $some_path/my_app_dependencies in the OS search path or loads libz.dll prior to initializing NVIGI which then results in NVIGI using incorrect libz.dll which can lead to instability or incorrect runtime behavior.

$some_path/nvigi/core
│
├──nvigi.core.framework.dll
├──cudart64_12.dll
└──libz.dll // !!! NVIGI version

$some_path/my_app_dependencies
│
└──libz.dll // !!! HOST version

NOTE: If similar issue occurs while deploying your application please contact NVIDIA

Troubleshooting

Crash Dumps and Exceptions

If unhandled exception is thrown in a NVIGI module, the crash dump and log file will be saved under ProgramData/NVIDIA/NVIGI/CrashDumps/$EXE_NAME/$ID by default.

Here is an example of the exception file structure:

C:\\ProgramData\\NVIDIA\\NVIGI\\CrashDumps\\nvigi.test\\1738350995226850
│
├──nvigi-log.txt
└──nvigi-sha-e46f10d.dmp

The crash callstack is shown at the end of the log file, providing a detailed trace of the function calls leading to the exception. Developers can use this information to identify the source of the issue.

NOTE: To capture as much as detail as possible tne verbose logging should be enabled and path to a log file must be provided when calling nvigiInit

For example:

[2025-01-31 11:16:35.227][nvigi][error][exception.cpp:322][writeMiniDump]Exception detected - thread 72668 - creating mini-dump 'C:/ProgramData/NVIDIA/NVIGI/CrashDumps/nvigi.test/1738350995226850/nvigi-sha-e46f10d.dmp'
[2025-01-31 11:16:41.917][nvigi][info][exception.cpp:377][writeMiniDump]Stack trace:
ntdll:RtlGetCurrentServiceSessionId:7fff4f92c37d
ntdll:RtlFreeHeap:7fff4f92b001
ucrtbase:free_base:7fff4d42364b
nvigi.core.framework:nvigiShutdown:7fff14560bb9
nvigi.test:C_A_T_C_H_T_E_S_T_134:7ff75782132a
nvigi.test:Catch::RunContext::invokeActiveTestCase:7ff757845619
nvigi.test:Catch::RunContext::runCurrentTest:7ff757856a4f
nvigi.test:Catch::RunContext::runTest:7ff757869cc1
nvigi.test:Catch::`anonymous namespace'::TestGroup::execute:7ff75783fb0b
nvigi.test:Catch::Session::runInternal:7ff757863b54
nvigi.test:Catch::Session::run:7ff757854e57
nvigi.test:wmain:7ff75788b415
nvigi.test:__scrt_common_main_seh:7ff75789c6dc
KERNEL32:BaseThreadInitThunk:7fff4dcc259d
ntdll:RtlUserThreadStart:7fff4f94af38

NOTE: Symbols can be found in the nvigi_pack which can be downloaded from https://developer.nvidia.com/rtx/in-game-inferencing