Prerequisites

This section covers the prerequisites for deploying and using the VSS Blueprint.

Hardware Requirements

The VSS Blueprint has been validated and tested on the following NVIDIA GPUs:

• NVIDIA H100
• NVIDIA GB300
• NVIDIA RTX PRO 6000 Blackwell
• NVIDIA L40S
• NVIDIA DGX SPARK
• NVIDIA IGX Thor
• NVIDIA AGX Thor
• NVIDIA RTX PRO 4500 Blackwell (limited support to alerts profile)

For GPUs that are not validated and tested, the OTHER configuration is available (experimental). See the deployment steps in each workflow guide for the GPU tab options.

If you need to know which platforms support your use-case, see the Performance section for benchmark results across models, use cases, and GPU platforms.

Software Requirements

The following software must be installed on your system:

• OS:

  • x86 hosts: Ubuntu 24.04
  • DGX Station (GB300): DGX OS 7.6.0
  • DGX-SPARK: DGX OS 7.5.0
  • IGX-THOR (T5000 and T7000): IGX-SW 2.0 Production (LTS), including IGX OS 2.0 (Ubuntu 24.04)
  • Jetson AGX-THOR: JetPack 7.2 (Jetson Linux/L4T r39.2)
  • Jetson AGX Orin and Jetson Orin NX: public JetPack 7.2.2 release

• NVIDIA Driver:

  • 595.58.03 (x86 hosts with Ubuntu 24.04)
  • 595.58.03 (DGX Station GB300)
  • 580.173.02 (DGX-SPARK)
  • 580.00, bundled with IGX-SW 2.0 (IGX-THOR)
  • 595.78 (Jetson AGX Orin, AGX-THOR, and Jetson Orin NX)

• NVIDIA Container Toolkit: 1.17.8+

• Docker: 28.3.3+ and earlier than 29.5.0

• Docker Compose: v2.39.1+

• Docker Buildx: 0.23.0 (working version); 0.23.0 or later and earlier than 0.35.0

• NGC CLI: 4.10.0+

• Python runner for Build Vision AI: uv is required for the Skill’s mandatory service-graph checks and helper scripts.

Install uv before starting the coding agent:

curl -LsSf https://astral.sh/uv/install.sh | sh

Build Vision AI runs the NemoClaw setup notebook non-interactively with a Python 3.12 environment and notebook packages supplied by uv. The pinned NemoClaw installer installs a compatible Node.js and npm runtime. A Jupyter kernel with Python 3.11 or later is required only when you follow the interactive NemoClaw notebook guide.

Runtime Environment Settings

In addition to the above prerequisites, the following runtime environment settings are required:

Linux Kernel Settings

# Ensure the sysctl.d directory exists
sudo ls -l /etc/sysctl.d
# Write VSS kernel settings to 99-vss.conf (persistent across reboots)
sudo bash -c "printf '%s\n' \
  'net.ipv6.conf.all.disable_ipv6 = 1' \
  'net.ipv6.conf.default.disable_ipv6 = 1' \
  'net.ipv6.conf.lo.disable_ipv6 = 1' \
  'net.core.rmem_max = 5242880' \
  'net.core.wmem_max = 5242880' \
  'net.ipv4.tcp_rmem = 4096 87380 16777216' \
  'net.ipv4.tcp_wmem = 4096 65536 16777216' \
  > /etc/sysctl.d/99-vss.conf"
# Reload sysctl to apply the new settings
sudo sysctl --system

Power Mode (only on IGX-THOR and AGX-THOR)

# Set max power mode
# NOTE: the below command requires a reboot to take effect
sudo nvpmodel -m 0

After reboot:

# Set max clocks
sudo jetson_clocks

Cache cleaner (only on DGX-SPARK, IGX-THOR and AGX-THOR)

Run the cache cleaner script for DGX SPARK and Jetson Thor.

Create the cache cleaner script at /usr/local/bin/sys-cache-cleaner.sh:

sudo tee /usr/local/bin/sys-cache-cleaner.sh << 'EOF'
#!/bin/bash
# Exit immediately if any command fails
set -e
# Disable hugepages
echo "disable vm/nr_hugepage"
echo 0 | tee /proc/sys/vm/nr_hugepages
# Notify that the cache cleaner is running
echo "Starting cache cleaner - Running"
echo "Press Ctrl + C to stop"
# Repeatedly sync and drop caches every 3 seconds
while true; do
	sync && echo 3 | tee /proc/sys/vm/drop_caches > /dev/null
	sleep 3
done
EOF
sudo chmod +x /usr/local/bin/sys-cache-cleaner.sh

Running in the background

sudo -b /usr/local/bin/sys-cache-cleaner.sh

Host firewall and Docker bridge subnets

If a default-deny firewall is enabled on the host, allow inbound traffic from the Docker bridge subnets used by VSS. Otherwise, bridge-networked containers cannot reach host-networked services and the affected workflow silently produces no output. See Bridge-network container cannot reach a VSS service on the host for diagnostics and the fix.

Minimum System Requirements

• 18 core CPU (x86 systems)
• 128 GB RAM
• 1 TB SSD
• 1 x 1 Gbps network interface
• NVIDIA GPU capacity required by the selected development profile. Use the Development Profile GPU Requirements tables below; validated layouts range from one to four GPUs depending on the profile, hardware, and model placement.

Network and Browser Access

The machine running the web browser must be able to reach TCP port 7777 on the deployment host. This port serves the VSS UI and routes optional web applications included in the selected service graph, such as Kibana and Phoenix. The default Build Vision AI deployment does not include Phoenix or the in-stack VSS Agent; the manual base-profile deployment includes both.

For cloud deployments, configure the instance firewall or security group to allow inbound TCP traffic on port 7777 from the browser client. For local deployments, ensure http://localhost:7777/ is reachable from the deployment host.

NGC + Hugging Face API Keys

The credentials you need depend on the images, models, and optional agent harness selected for the deployment. Standard VSS deployment paths require an NGC Personal API Key. A Hugging Face token is required only when a selected model or helper downloads gated or rate-limited weights from Hugging Face. See the credential decision table before deploying.

Create an NVIDIA NGC Account

1. Go to https://ngc.nvidia.com
2. Click Sign up if you don’t have an account, or Sign in if you do
3. Complete the registration process with your email address

Generate an API Key

1. Once logged in, click on your username in the top right corner
2. Select Setup from the dropdown menu
3. Navigate to API Keys under the Keys/Secrets section
4. Click Generate API Key
5. Click Generate Personal Key (available at both the top-center and top-right of the page)
6. Provide a descriptive name for your API key (e.g., “VSS Blueprint Development”)
7. Select NGC Catalog for Key Permissions
8. Click Generate Personal Key
9. Copy the generated API key immediately - you won’t be able to see it again
10. Store it securely in a password manager or encrypted file

Verify Your NGC Access

To verify your NGC key has the correct permissions:

# Set your NGC API key
export NGC_CLI_API_KEY='your_ngc_api_key'
# Authenticate Docker for gated NIM and other nvcr.io images. The username is
# the literal string $oauthtoken; the password is your NGC Personal API Key.
printf '%s' "$NGC_CLI_API_KEY" | \
  docker login nvcr.io --username '$oauthtoken' --password-stdin
# Test access to the required resources
ngc registry resource list "nvidia/vss-developer/*"
ngc registry resource list "nvidia/vss-core/*"
# Test access to the required images
ngc registry image list "nvidia/vss-core/*"
# Test access to the required models
ngc registry model list "nvidia/tao/*"

If you encounter permission errors, contact NVIDIA support for assistance.

The Login Succeeded response verifies NGC Docker registry authentication. Run this step before the Build Vision AI Skill or any manual docker compose up command that pulls gated NIM or other nvcr.io images. A later docker compose config --quiet validates that Compose can render the model, but a successful exit does not prove that every operationally required environment value is set. Review its unset-variable warnings against the selected profile. The command does not contact the registry or validate image entitlements.

Hugging Face Access Token

Some optional model selections download weights from Hugging Face. Create a token only when the selected deployment path identifies HF_TOKEN as required. Public downloads can also be rate limited without a token.

1. Log in to Hugging Face
2. Go to Access Tokens and create a token with Read permission
3. Export the token before deploying:

export HF_TOKEN='hf_your_token_here'

Credentials used by the Build Vision AI Skill

The vss-build-vision-ai Skill asks for credentials before it deploys. Which keys are required depends on model placement and whether you choose the optional NemoClaw harness.

Keep provider credentials out of chat transcripts and generated build artifacts. For a custom OpenAI-compatible endpoint, put the provider-issued key in the checkout-root, gitignored nemoclaw.env file, or export it in the shell that starts the coding-agent session:

COMPATIBLE_API_KEY='your-provider-api-key'

The Skill loads nemoclaw.env before harness setup. If you export the variable instead, restart the coding-agent session from that shell so it inherits the credential.

A self-hosted OpenAI-compatible harness endpoint that does not require a real token can use the documented placeholder. Selecting no harness requires no harness credential. These choices do not remove the NGC or model-download credentials required by the VSS stack itself. The Skill stops before deployment and identifies any missing credential; it does not silently switch providers.

Development Profile GPU Requirements

The following tables show the number of GPUs needed for each development profile:

References

¹ GPU device ‘0’ is needed to run alerts.

² GPU devices ‘0’ and ‘1’ are needed to run search.

VLM Custom Weights

The VSS Blueprint supports using VLM custom weights for specialized use cases. This section explains how to download custom weights from NGC or Hugging Face.

Download Custom Weights from NGC

If VLM custom weights are available on NGC, you can download them using the NGC CLI:

# Set your NGC API key if not already set
export NGC_CLI_API_KEY='your_ngc_api_key'
# Download custom weights from NGC
ngc registry model download-version <org>/<team>/<model>:<version>
# Move downloaded custom weights to desired path/folder
mv </downloaded/path/in/ngc/output> </path/to/custom/weights>

The weights will be downloaded to a local directory. Note the path to this directory, then set VLM_CUSTOM_WEIGHTS in the selected profile’s *-overrides.env file before deploying (the deployment guides use user-overrides.env as an example).

Download Custom Weights from Hugging Face

To download custom VLM weights from Hugging Face, you can use the huggingface-cli tool or Git LFS:

Using huggingface-cli:

# Install Hugging Face CLI if not already installed
pip install -U "huggingface_hub[cli]"
# Login to Hugging Face (required for gated models)
huggingface-cli login
# Create a base directory where the custom weights can be downloaded
mkdir -p </path/to/custom/weights>
# Download a model
huggingface-cli download <model-id> --local-dir </path/to/custom/weights>

Using Git LFS:

# Install Git LFS if not already installed
sudo apt install git-lfs
git lfs install
# Clone the model repository
git clone https://huggingface.co/<model-id> </path/to/custom/weights>

Verify Downloaded Weights

After downloading custom weights, verify the directory structure and contents:

# List the contents of the weights directory
ls -lh </path/to/custom/weights>

Typical VLM weight directories contain:

• Model configuration files (e.g., config.json)
• Model weights (e.g., pytorch_model.bin, model.safetensors, or sharded weights)
• Tokenizer files (e.g., tokenizer.json, tokenizer_config.json)
• Other metadata files

Common Installation Issues

When pulling resources from NGC, you may encounter the following issues:

Error: Missing org - If Authenticated, org is also required.

Steps to resolve:

1. Generate an NGC API key with your desired org selected.
2. Ensure that the org you selected in ngc config set is the same as the org you selected in the NGC API key generation.

Appendix: Software Prerequisites Installation Guide

Install NVIDIA Driver

Ubuntu 24.04

NVIDIA Driver version 595.58.03 is required with Ubuntu 24.04.

You can download the driver directly from: https://www.nvidia.com/en-us/drivers/details/266021/

Additional resources

For detailed installation instructions, refer to the official NVIDIA Driver Installation Guide: https://docs.nvidia.com/datacenter/tesla/driver-installation-guide/index.html

You can also browse drivers for your specific GPU and platform from the NVIDIA Driver Downloads page: https://www.nvidia.com/Download/index.aspx

# Install the Fabric Manager package that matches driver 595.58.03
sudo apt-get update
sudo apt-get install -y nvidia-fabricmanager-595=595.58.03-1ubuntu1
# Enable and start the service
sudo systemctl enable nvidia-fabricmanager
sudo systemctl start nvidia-fabricmanager
# Verify the service is running
sudo systemctl status nvidia-fabricmanager

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/nvidia-fabricmanager_595.58.03-1ubuntu1_amd64.deb
sudo dpkg -i nvidia-fabricmanager_595.58.03-1ubuntu1_amd64.deb

Install Docker

Docker version 28.3.3 or later and earlier than 29.5.0 is required. If Docker is not installed, first install Docker Engine from the official Ubuntu apt repository. The latest package is not necessarily in the VSS-tested range.

Run docker without sudo

After installation, complete the Linux post-installation steps so that Docker can run without sudo. See Linux post-installation steps for Docker Engine.

Pin and configure Docker

From the VSS repository root, run the checked-in setup script before starting any containers:

bash deploy/docker/scripts/pin_docker_version.sh
docker version --format '{{.Server.Version}}'
docker info --format '{{.CgroupDriver}}'

The script installs the repository’s known-good Docker Engine, Compose, Buildx, and containerd package versions when the installed engine is outside the tested range. It holds the applicable packages and safely merges the required native.cgroupdriver=cgroupfs setting into /etc/docker/daemon.json. It is idempotent and skips the package change when Docker is already in range.

Install Docker Compose

Docker Compose v2.39.1 or later is required. Docker Compose v2 is typically installed as a plugin with modern Docker installations.

Check if Docker Compose is already installed and meets the minimum version:

docker compose version

The default docker-compose-plugin package in Ubuntu repositories may provide a version older than v2.39.1. If the version is missing or too old, select the binary for the host architecture and install the required version manually:

case "$(uname -m)" in
  x86_64) COMPOSE_ARCH=x86_64 ;;
  aarch64|arm64) COMPOSE_ARCH=aarch64 ;;
  *) echo "Unsupported architecture: $(uname -m)" >&2; exit 1 ;;
esac
sudo mkdir -p /usr/local/lib/docker/cli-plugins
sudo curl -fSL "https://github.com/docker/compose/releases/download/v2.39.1/docker-compose-linux-${COMPOSE_ARCH}" \
  -o /usr/local/lib/docker/cli-plugins/docker-compose
sudo chmod +x /usr/local/lib/docker/cli-plugins/docker-compose

Verify installation:

docker compose version

Install Docker Buildx

Buildx version 0.23.0 or later and earlier than 0.35.0 is required. Version 0.23.0 is the known working version.

Check if Docker Buildx is already installed and meets the version range:

docker buildx version

If the version is missing, older than 0.23.0, or 0.35.0 or newer, install working version 0.23.0:

case "$(uname -m)" in
  x86_64) BUILDX_ARCH=amd64 ;;
  aarch64|arm64) BUILDX_ARCH=arm64 ;;
  *) echo "Unsupported architecture: $(uname -m)" >&2; exit 1 ;;
esac
sudo mkdir -p /usr/local/lib/docker/cli-plugins
sudo curl -fSL "https://github.com/docker/buildx/releases/download/v0.23.0/buildx-v0.23.0.linux-${BUILDX_ARCH}" \
  -o /usr/local/lib/docker/cli-plugins/docker-buildx
sudo chmod +x /usr/local/lib/docker/cli-plugins/docker-buildx

Verify installation:

docker buildx version

Install NVIDIA Container Toolkit

The NVIDIA Container Toolkit is required to run the NVIDIA containers. Follow the guide here for installing the NVIDIA Container Toolkit: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

Install NGC CLI

NGC CLI version 4.10.0 or later is required to download blueprints and authenticate with NGC resources.

Download and install NGC CLI:

For ARM64 Linux:

curl -sLo "/tmp/ngccli.zip" https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/4.10.0/files/ngccli_arm64.zip

For AMD64 Linux:

curl -sLo "/tmp/ngccli.zip" https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/4.10.0/files/ngccli_linux.zip

After downloading, install NGC CLI:

sudo mkdir -p /usr/local/bin
sudo unzip -qo /tmp/ngccli.zip -d /usr/local/lib
sudo chmod +x /usr/local/lib/ngc-cli/ngc
sudo ln -sfn /usr/local/lib/ngc-cli/ngc /usr/local/bin/ngc

Verify installation:

ngc --version

Configure NGC CLI with your API key:

ngc config set

When prompted, enter your NGC API key. For information on obtaining API keys, see NGC + Hugging Face API Keys.

Setup DGX-SPARK

For setup instructions, see the DGX Spark User Guide.

Setup installs DGX OS (including the NVIDIA driver), Docker with NVIDIA Container Toolkit integration, and NGC access.

DGX Spark ships with DGX OS pre-installed; do not flash a separate OS image for normal setup. VSS deployments require DGX OS 7.5.0, NVIDIA driver 580.173.02.

Connect the display, keyboard, mouse, and network before connecting power. The system powers on as soon as the power supply is connected. Complete the OOBE wizard and allow its software download and automatic reboot to finish; do not power off or reboot while software is being installed.

Setup AGX-THOR

Setup JetPack 7.2 (Jetson Linux/L4T r39.2) via instructions in the Jetson AGX Thor Developer Kit User Guide. Use the Jetson ISO r39.2 USB installation path and select NVMe as the target storage.

• Create the bootable USB with BalenaEtcher; copying the ISO files to a USB drive is not sufficient.
• Accept the QSPI update when prompted. Do not skip this update.
• After installation, remove the USB drive before rebooting, complete oem-config, and install JetPack with sudo apt install nvidia-jetpack.
• The resulting stack must report Jetson Linux/L4T r39.2, NVIDIA driver 595.78.

Setup IGX-THOR

Setup IGX-SW 2.0 Production (LTS), including IGX OS 2.0, via instructions in the Jetson IGX Thor User Guide.

• Setup installs NVIDIA driver 580.00 with the IGX-SW 2.0 stack.
• For T5000, install from a bootable USB and select Install IGX OS with Real-Time Kernel.
• For T7000, use a bootable USB or BMC Virtual Media. Update the firmware in the BMC WebUI before installing, and connect the monitor to the iGPU DisplayPort rather than the dGPU port.
• After installing either platform, install the IGX AI software with sudo apt install nvidia-igx-ai and enable the NVIDIA CDI service as described in the user guide.

NIM configuration settings

The deployment ships with NIM (NVIDIA Inference Microservice) configuration via environment variables for supported NIMs and GPUs. You can use the default env files, rely on NIM defaults for unsupported GPUs, or supply your own env files at install time.

Default configuration

For each supported NIM and GPU, the deployment includes a corresponding env file under ./deploy/docker/services/nim/<nim-name>/. For example, for the cosmos3-reasoner NIM on an NVIDIA RTX PRO 6000 Blackwell:

./deploy/docker/services/nim/cosmos3-reasoner/hw-RTXPRO6000BW.env

Unsupported GPU hardware

For GPU hardware outside the listed profiles, set HARDWARE_PROFILE=OTHER in the selected profile’s *-overrides.env file. The deployment uses a default empty env file and the NIM falls back to its own defaults.

./deploy/docker/services/nim/cosmos3-reasoner/hw-OTHER.env

Custom env files at install time

You can define your own env variants for a NIM in the selected profile’s *-overrides.env file. Set LLM_ENV_FILE and/or VLM_ENV_FILE; paths can be absolute or relative to deploy/docker.

LLM_ENV_FILE=/path/to/llm.env
VLM_ENV_FILE=/path/to/vlm.env