Prerequisites#

This page outlines the hardware and software requirements for running NVIDIA NIM for Cosmos.

Hardware Requirements#

  • GPU(s): NVIDIA NIM for Cosmos requires NVIDIA GPUs with Ampere architecture or later. The GPUs must have sufficient memory—refer to the Supported Models section for specific memory requirements by model.

    • This NIM can also run on multiple homogeneous NVIDIA GPUs with sufficient aggregate memory and a CUDA compute capability of 7.0 or later (8.0 for bfloat16).

  • CPU: Currently, only x86_64 architecture is supported.

  • CPU Memory: At least 90GB of RAM is required.

  • Disk Space: At least 100GB of disk space is required for the container and model.

Software Requirements#

  • Operating System: Any Linux distribution that meets the following criteria:

  • NVIDIA GPU Drivers: Version 535 or later

  • CUDA Drivers: Follow the CUDA installation guide to install CUDA drivers.

    For CUDA installation, we recommend the following:

    • Use a network repository as part of a package manager installation and skip the CUDA toolkit installation, as the required libraries are available within the NIM container.

    • Install the open kernels for a specific version of CUDA:

      Major Version

      EOL Date

      Data Center and RTX/Quadro GPUs

      GeForce GPUs

      > 550

      TBD

      ✓

      ✓

      550

      Feb 2025

      ✓

      ✓

      545

      Oct 2023

      ✓

      ✓

      535

      June 2026

      ✓

      525

      Nov 2023

      ✓

      470

      Sept 2024

      ✓

  • Docker: Version 23.0.1 or later. Follow the Docker installation guide.

  • NVIDIA Container Toolkit: Version 1.16.2 or later. Follow the NVIDIA Container Toolkit installation guide.

Configuration and Verification#

After installing the NVIDIA Container Toolkit, configure Docker by following the instructions in the Configure Docker section of the NVIDIA Container Toolkit documentation.

To verify your setup, run the following command:

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

This command should produce output similar to the following, confirming your CUDA driver version and available GPUs:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 80GB HBM3          On  |   00000000:1B:00.0 Off |                    0 |
| N/A   36C    P0            112W /  700W |   78489MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Tip

Refer to the troubleshooting page if you are encountering issues during the verification step.