Loading Custom and Fine-Tuned Evo 2 NIM Model Checkpoints

Prerequisites

Before proceeding, ensure the following prerequisites are met:

• Your model has been trained or fine-tuned using the NVIDIA BioNeMo Framework.
• The model is saved in the NeMo format (the default output format of the BioNeMo Framework).

Specifying a Custom Evo 2 NIM Model Checkpoint

To configure NIM to load a custom checkpoint, follow these steps:

1. Disable automatic model downloading

   Set the NIM_DISABLE_MODEL_DOWNLOAD environment variable to 1. This prevents NIM from attempting to download the default model from NGC.

2. Mount and specify the model path

   Mount your custom checkpoint directory into the Docker container and set the MODEL_PATH environment variable inside the container so that it points to the mounted directory.

Example: Loading a Custom or Manually Downloaded Evo 2 Model

You can test this process by manually downloading the original Evo 2 7B 1M model and instructing NIM to load it as a custom checkpoint, even if you have not performed fine-tuning.

1. Set the required variables and paths:

   MODEL=evo2-7b:2.0.0
   MODEL_PATH="$PWD/$(echo "$MODEL" | sed s/:/_v/)"

2. Download the model:

   ngc registry model download-version nim/arc/$MODEL

3. Run NIM with the appropriate mount points and environment variables:

   docker run -it \
       --runtime=nvidia \
       --gpus='"device=0,1"' \
       -p 8000:8000 \
       -v "$MODEL_PATH":"$MODEL_PATH":ro \
       -e MODEL_PATH="$MODEL_PATH" \
       -e NIM_DISABLE_MODEL_DOWNLOAD=1 \
       nvcr.io/nim/arc/evo2:2

This sequence of commands starts the NIM container and loads your custom model checkpoint. The example downloads the 7B model, which the hardware table lists as a single-GPU configuration. For additional examples, refer to the Quickstart Guide for Evo 2 NIM.