Loading Custom and Fine-Tuned Evo 2 NIM Model Checkpoints#

Prerequisites#

Before proceeding, ensure the following prerequisites are met:

  • Your model has been trained or fine-tuned using the NVIDIA BioNeMo Framework.

  • The model is saved in the NeMo format (the default output format of the BioNeMo Framework).

Specifying a Custom Evo 2 NIM Model Checkpoint#

To configure NIM to load a custom checkpoint, follow these steps:

  1. Disable automatic model downloading

    Set the NIM_DISABLE_MODEL_DOWNLOAD environment variable to 1. This prevents NIM from attempting to download the default model from NGC.

  2. Mount and specify the model path

    Mount your custom checkpoint directory into the Docker container and set the MODEL_PATH environment variable inside the container pointing it to the mounted directory.

Example: Loading a Custom or Manually Downloaded Evo 2 Model#

You can test this process by manually downloading the original Evo 2 40B 1M model and instructing NIM to load it as a custom checkpoint, even if you have not performed fine-tuning.

  1. Set the required variables and paths:

    MODEL=evo2-40b:2.0.0
MODEL_PATH="$PWD/$(echo "$MODEL" | sed s/:/_v/)"

  2. Download the model:

    ngc registry model download-version nvstaging/nim/$MODEL

  3. Run NIM with the appropriate mount points and environment variables:

    docker run -it \
    --runtime=nvidia \
    --gpus='"device=0,1"' \
    -p 8000:8000 \
    -v "$MODEL_PATH":"$MODEL_PATH":ro \
    -e MODEL_PATH="$MODEL_PATH" \
    -e NIM_DISABLE_MODEL_DOWNLOAD=1 \
    nvcr.io/nim/arc/evo2-40b:2

This sequence of commands will start the NIM container and load your custom model checkpoints. For additional examples and information, refer to the Quickstart Guide for Evo 2 NIM.