The Stable Diffusion 3D Sample

The standalone Stable Diffusion 3D sample, nvigi.3d.sd, combines NVIGI Stable Diffusion plugins with the Donut renderer. It presents the Sponza scene together with an asset editor that can generate and apply textures while rendering continues. The existing nvigi.3d sample remains the ASR, GPT, and TTS conversational sample.

Launching the Sample

These instructions use the following path placeholders, which refer to locations that differ between a binary developer pack and a GitHub source tree:

<SDK_ROOT>: the root of the SDK Plugins tree, which contains the bin directory for the SDK.
<SDK_PLATFORM>: the target CPU architecture. See Platform Support for information on supported platforms.

The executable is located at <SDK_ROOT>/bin/<SDK_PLATFORM>/<Configuration>/nvigi.3d.sd.exe. Direct3D 12 is the default renderer:

cd <SDK_ROOT>\bin\<SDK_PLATFORM>\<Configuration>
nvigi.3d.sd.exe

Use -vk to render with Vulkan:

nvigi.3d.sd.exe -vk

The sample reuses the Sponza scene, fonts, and other rendering media in <SDK_ROOT>/data/nvigi.test/nvigi.3d; there is no separate copy of the 3D media for this sample. The application searches upward from the executable for that directory.

Models and Backends

At startup, the sample searches upward from the executable for <SDK_ROOT>/data/nvigi.models. The model root must contain the model payload directory nvigi.plugin.sd.ggml directly beneath it. A configs/nvigi.plugin.sd.ggml directory contains model cards only and is not sufficient by itself. For a non-standard layout, specify the model root explicitly:

For a generated local SDK tree, populate the model payloads before launching:

<SDK_ROOT>\download_data.bat

nvigi.3d.sd.exe -pathToModels D:\models\nvigi.models

By default, the sample automatically selects and loads a compatible local image model, preferring a Flux model when one is available. It enumerates the CUDA backend and the inference backend matching the renderer (D3D12 or Vulkan). Open Options… to:

Switch between automatic and manual backend selection.
Select the image-generation model and backend.
Set the automatic-selection VRAM budget.
Store Stable Diffusion parameters on the CPU.
Select a separate material-map model for optional roughness and metallic generation.

Use -noLoad to skip the initial model load. A model can then be selected or loaded from the UI.

Editing Scene Assets

The SD Asset Editor lists textured materials from the loaded scene. To edit an asset:

Select an asset and enter a prompt. Selection applies its saved camera view when one exists; otherwise, the camera remains unchanged.
Choose prompt-only generation (txt2img) or, when supported by the selected model, use the existing texture as an image-to-image reference.
Optionally adjust strength, steps, CFG scale, and seed under Advanced Settings.
Select Generate & Apply. Enable Apply prompt to all assets to process the complete list instead.

The generated image replaces the selected material’s base/diffuse texture in the running scene. The editor keeps the original texture available for comparison.

Enable Replace roughness/metallic maps (Marigold) to generate material maps in addition to the base texture. Select a compatible material-map model in Options… first. After generation, Use generated roughness/metallic maps switches between the generated maps and the material’s original maps.

Select Revert to Original to restore the selected material’s original base texture and material maps. These edits are applied in memory; the source Sponza files are not overwritten.

Camera View Presets

Each asset can have a predefined camera position stored in sd-camera-poses.json. When you click an asset, the camera automatically moves to face it so you can clearly see which asset is being updated. If no position is defined for that asset, the camera stays where it is.

The presets are shipped in data/nvigi.test/nvigi.3d/sd-camera-poses.json and loaded read-only with the scene data.

Useful Command-Line Options

Argument	Effect
`-vk`	Use Vulkan rendering. Direct3D 12 is the default.
`-pathToModels <path>`	Use the specified `nvigi.models` root.
`-noLoad` or `-skipLoad`	Do not load the selected model at startup.
`-noCIG`	Disable CUDA in Graphics (CiG) integration.
`-logToFile <directory>`	Write NVIGI logs to the specified existing directory.
`-noSigCheck`	Disable NVIGI DLL signature verification.
`-width <pixels>` / `-height <pixels>`	Set the initial window dimensions.
`-vsync`	Enable vertical synchronization.
`-fullscreen`	Start in full-screen mode.
`-debug`	Enable the graphics debug runtime and NVRHI validation.
`-verbose`	Enable informational Donut logging.
`-maxFrames <count>`	Exit after the specified number of rendered frames.