Harmonizer NIM Prerequisites | NVIDIA NIM Harmonizer

This page outlines the hardware and software requirements for running Harmonizer NIM.

Hardware Requirements

Confirm the following hardware requirements:

GPU: One supported NVIDIA GPU per container. The smallest qualified configuration is specifically the 24 GB GeForce RTX 4090; this does not imply support for every GPU with 24 GB of memory. The supported 48 GB and 80 GB-or-larger configurations use progressively larger startup specialization sets. Refer to the Support Matrix for the exact GPU SKUs.
CPU architecture: x86_64 (amd64) and ARM64 are supported.
CPU memory: Allocate at least 32 GiB of host memory per container for the recommended configuration.
Disk space: Reserve 50 GiB of free space before the first pull. This accommodates the unpacked image, model checkpoint, generated compilation cache, compressed layers retained temporarily while Docker unpacks the image, and operating headroom.

Release-Candidate Sizing Reference

The following measurements were collected with a pre-release container and can differ from the published 1.0.0 image. Sizes are binary GiB. The runtime measurements used an L40S on an x86_64 host, an empty compilation cache, cold startup, readiness, and one 1024x576 HTTP inference request.

Item	Measured Size
Compressed registry download, amd64	9.82 GiB
Compressed registry download, arm64	10.49 GiB
Unpacked Docker image, amd64	22.89 GiB
Release model checkpoint	4.70 GiB
Generated compilation cache after readiness and inference	0.45 GiB
Peak container CPU memory during cold startup and inference	10.23 GiB
Passing Docker memory limit in one controlled L40S run	16.00 GiB, swap disabled
Peak L40S GPU memory during cold startup and inference	35.4 GiB

If Docker’s data root and the persistent NIM cache are on separate file systems, budget at least 35 GiB for Docker and 10 GiB for the NIM cache. The model and generated TorchInductor and Triton artifacts are stored in the persistent cache mounted at /opt/nim/.cache.

Software Requirements

Install the following software:

Operating system: A 64-bit Linux distribution supported by NVIDIA Container Toolkit. The Harmonizer image is published for linux/amd64 and linux/arm64 and uses Ubuntu 24.04 container userspace.
NVIDIA GPU driver: Version 595.58.03 or later. The image bundles CUDA 13.2.1.009 and performs TorchInductor and Triton PTX compilation during cold startup. For this reason, use the corresponding CUDA 13.2 Update 1 driver baseline instead of relying on the broader CUDA 13.x minor-compatibility floor. Refer to the CUDA Toolkit release notes and minor-version compatibility limitations.
CUDA Toolkit: Not required on the host. The container includes the CUDA runtime and libraries. Refer to the NVIDIA PyTorch 26.05 release notes for the bundled runtime stack.
Container runtime: Docker 23.0.1 or later, or a compatible OCI runtime that meets the NVIDIA NIM platform requirements. Follow the Docker installation guide.
NVIDIA Container Toolkit: Version 1.16.2 or later. Follow the NVIDIA Container Toolkit installation guide.

Verified Configurations

The release-candidate validation run used Ubuntu 24.04.4 on x86_64, NVIDIA driver 595.58.03, Docker 29.6.1, and NVIDIA Container Toolkit 1.19.1. These versions are a validated configuration, not higher minimums than those listed above.

Configuration and Verification

After installing the NVIDIA Container Toolkit, configure Docker by following the NVIDIA Container Toolkit documentation.

To verify that Docker can access the GPU, run:

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

The command should list the available NVIDIA GPUs and driver version.

Next Steps

After checking these requirements and verifying GPU access, follow the Quickstart to launch the NIM.