Prerequisites and Support Matrix for Evo 2 NIM | NVIDIA BioNeMo NIM Documentation

Models

Model Name	Parameters	Context Length	Publisher
Evo 2	40 Billion	1 Million	Arc Institute
Evo 2	7 Billion	1 Million	Arc Institute

Hardware Support

Run the Evo 2 NIM on an NVIDIA GPU that meets the following hardware requirements:

40B Model (Default)

GPU	GPU Memory (GB)	Precision	# of GPUs	Disk Space (GB)	CPU RAM (GB)
NVIDIA H100 80GB HBM3	80	Mixed	2	110	16
NVIDIA H200	141	Mixed	1	110	16

7B Model

GPU	GPU Memory (GB)	Precision	# of GPUs	Disk Space (GB)	CPU RAM (GB)
NVIDIA H100 80GB HBM3	80	Mixed	1	50	16
NVIDIA H200	141	Mixed	1	50	16
NVIDIA RTX 6000 Ada Generation	48	Mixed	1	50	16
NVIDIA L40S	48	Mixed	1	50	16

The GPU Memory, CPU RAM, and Disk Space values are in GB; Disk Space is for both the container and the model.

If the precision is mixed, it means the model features layers with FP32, BF16, and FP8 floating-point precision tensors.

Supported CPU architectures: x86_64 / amd64.

Software Prerequisites

Begin with Linux-based Docker-supported operating system
Install Docker - minimum version: 23.0.1
Install NVIDIA Drivers - minimum version: 550.90.07
Install the NVIDIA Container Toolkit - minimum version: 1.13.5
Verify your container runtime supports NVIDIA GPUs by running

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

Example output:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 80GB HBM3          On  |   00000000:1B:00.0 Off |                    0 |
| N/A   33C    P0             75W /  700W |       1MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA H100 80GB HBM3          On  |   00000000:43:00.0 Off |                    0 |
| N/A   36C    P0             70W /  700W |       1MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

For more information on enumerating multi-GPU systems, refer to the NVIDIA Container Toolkit GPU enumeration topic.

NGC Account

Create an account on NGC
Generate an API key
Log in to Docker with your NGC API key. Enter the key as the password when prompted.

docker login nvcr.io --username='$oauthtoken'