Prerequisites and Support Matrix for Evo 2 NIM#

Models#

Model Name	Parameters	Context Length	Publisher
Evo 2	40 Billion	1 Million	Arc Institute

Hardware Support#

Run your Evo 2 NIM on any NVIDIA GPU that meets following minimum hardware requirements:

GPU	GPU Memory (GB)	Precision	# of GPUs	Disk Space (GB)	CPU RAM (GB)
H100	80	Mixed	2	100	16
H200	144	Mixed	1	100	16

The GPU Memory, CPU RAM, and Disk Space values are in GB; Disk Space is for both the container and the model.

If the precision is mixed, it means the model features layers with FP32, BF16, and FP8 floating-point precision tensors.

Supported CPU architectures: x86_64 / amd64.

Software Prerequisites#

Begin with Linux-based Docker-supported operating system
Install Docker - minimum version: 23.0.1
Install NVIDIA Drivers - minimum version: 550.90.07
Install the NVIDIA Container Toolkit - minimum version: 1.13.5
Verify your container runtime supports NVIDIA GPUs by running

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

Example output:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 80GB HBM3          On  |   00000000:1B:00.0 Off |                    0 |
| N/A   33C    P0             75W /  700W |       1MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA H100 80GB HBM3          On  |   00000000:43:00.0 Off |                    0 |
| N/A   36C    P0             70W /  700W |       1MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Note

For more information on enumerating multi-GPU systems, see the NVIDIA Container Toolkit’s GPU Enumeration Docs

NGC (NVIDIA GPU Cloud) Account#

Create an account on NGC
Generate an API Key
Log in to Docker with your NGC API (enter the key as password when prompted.)

docker login nvcr.io --username='$oauthtoken'