Is this page helpful?

Prerequisites#

This page outlines the hardware and software requirements for running NVIDIA NIM for Cosmos WFM (World Foundation Models).

Hardware Requirements for Predict1#

  • GPU(s): Predict1 requires NVIDIA GPUs with Ampere architecture or later. The GPUs must have sufficient memory—refer to the Supported Models section for specific memory requirements by model.

    • This NIM can also run on multiple homogeneous NVIDIA GPUs with sufficient aggregate memory and a CUDA compute capability of 7.0 or later (8.0 for bfloat16).

  • CPU: Currently, only x86_64 architecture is supported.

  • CPU Memory: At least 90GB of RAM is required.

  • Disk Space: At least 100GB of disk space is required for the container and model.

Software Requirements for Predict1#

  • Operating System: Any Linux distribution that meets the following criteria:

  • NVIDIA GPU Drivers: Version 535 or later

  • CUDA Drivers: Follow the CUDA installation guide to install CUDA drivers.

    For CUDA installation, we recommend the following:

    • Use a network repository as part of a package manager installation and skip the CUDA toolkit installation, as the required libraries are available within the NIM container.

    • Install the open kernels for a specific version of CUDA:

      Major Version

      EOL Date

      Data Center and RTX/Quadro GPUs

      GeForce GPUs

      > 550

      TBD

      ✓

      ✓

      550

      Feb 2025

      ✓

      ✓

      545

      Oct 2023

      ✓

      ✓

      535

      June 2026

      ✓

      525

      Nov 2023

      ✓

      470

      Sept 2024

      ✓

  • Docker: Version 23.0.1 or later. Follow the Docker installation guide.

  • NVIDIA Container Toolkit: Version 1.16.2 or later. Follow the NVIDIA Container Toolkit installation guide.

Hardware Requirements for Transfer2.5#

  • GPU(s): Transfer2.5 requires NVIDIA GPUs with Hopper architecture or later. The GPUs must have sufficient memory—refer to the Supported Models section for specific memory requirements by model.

    • This NIM can also run on multiple homogeneous NVIDIA GPUs with sufficient aggregate memory and a CUDA compute capability of 9.0 or later.

  • CPU: x86_64 and arm architectures are supported.

  • CPU Memory: At least 90GB of RAM is required.

  • Disk Space: At least 100GB of disk space is required for the container and model.

Software Requirements for Transfer2.5#

Hardware Requirements for Cosmos3-Generator#

  • GPU(s): Cosmos3-Generator requires NVIDIA GPUs with Hopper architecture or later (CUDA compute capability 9.0 or later) for bf16 and fp8 precision. nvfp4 precision additionally requires Blackwell architecture (CUDA compute capability 10.0 or later).

    • The GPUs must have sufficient memory — refer to Configurations for Cosmos3-Generator for the per-SKU compatibility grid and the supportability gates. As a rule of thumb, ≥ 80 GB of VRAM covers all 8B (nano) profiles; the 32B (super) size needs ≥ 150 GB VRAM for bf16 and ≥ 115 GB VRAM for fp8 profile for best performance. For super model there are also TP profiles, that decrease memory requirements to fit model into 80GB VRAM GPU but require 2-4 GPUs minimum and decreate overall perfomance.

    • Cosmos3-Generator can run on multiple homogeneous NVIDIA GPUs (1,2,4 or 8) with sufficient memory each and a CUDA compute capability of 9.0 or later (10.0 for nvfp4). Heterogeneous GPU mixes are not supported — within a single replica, all visible GPUs must be the same SKU.

  • CPU: x86_64 only. ARM (aarch64) is not currenlty supported by Cosmos3-Generator.

  • CPU Memory: At least 150 GB of RAM is required for the super model.

  • Disk Space: At least 150 GB of disk space is required for the container image and the downloaded model artifacts.

  • Shared Memory: At least 16 GB (--shm-size 16g) must be available to the container. The NIM stages intermediate image and video buffers under /dev/shm during inference; the Docker default of 64 MB is not enough. Use --shm-size 32g for multi-GPU configurations.

Software Requirements for Cosmos3-Generator#

  • Operating System: Any Linux distribution that meets the following criteria:

  • NVIDIA GPU Drivers: Version 580 or later.

  • CUDA Drivers: Follow the CUDA installation guide to install CUDA drivers.

    For CUDA installation, we recommend the following:

    • Use a network repository as part of a package manager installation and skip the CUDA toolkit installation, as the required libraries are available within the NIM container.

    • Install the open kernels for a specific version of CUDA:

      Major Version

      EOL Date

      Data Center and RTX/Quadro GPUs

      GeForce GPUs

      > 590

      TBD

      ✓

      ✓

      590

      Dec 2026

      ✓

      ✓

      580

      Aug 2028

      ✓

      ✓

  • Docker: Version 23.0.1 or later. Follow the Docker installation guide.

  • NVIDIA Container Toolkit: Version 1.16.2 or later. Follow the NVIDIA Container Toolkit installation guide.

  • NGC API key: A valid NGC_API_KEY with read access to the NGC container registry that hosts the Cosmos3-Generator image and to the model registry that hosts the weight artifacts. See Quickstart Guide for how to generate, export, and use the key.

Configuration and Verification#

After installing the NVIDIA Container Toolkit, configure Docker by following the instructions in the Configure Docker section of the NVIDIA Container Toolkit documentation.

To verify your setup, run the following command:

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

This command should produce output similar to the following, confirming your CUDA driver version and available GPUs:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 80GB HBM3          On  |   00000000:1B:00.0 Off |                    0 |
| N/A   36C    P0            112W /  700W |   78489MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Tip

Refer to the troubleshooting page if you are encountering issues during the verification step.