Quickstart Guide for Evo 2 NIM

Start NIM

1. Export the NGC_API_KEY variable.

export NGC_API_KEY=<your personal NGC key>

2. NIM container automatically downloads models. To save time and bandwidth it is recommended to provide local cache directory. This way NIM will be able to reuse already downloaded models. Execute following command to setup cache directory.

export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
sudo chmod 0777 -R "$LOCAL_NIM_CACHE"

3. Run the NIM container with the following commands.

docker run -it \
    --runtime=nvidia \
    --gpus='"device=0,1"' \
    -p 8000:8000 \
    -e NGC_API_KEY \
    -v "$LOCAL_NIM_CACHE":/opt/nim/.cache \
    nvcr.io/nim/arc/evo2-40b:1

This command will start the NIM container and expose port 8000 for the user to interact with the NIM.

4. Open a new terminal, leaving the terminal open with the just launched service. In the new terminal, wait until the health check end point returns {"status":"ready"} before proceeding. This may take a couple of minutes. You can use the following command to query the health check.

curl http://localhost:8000/v1/health/ready

DNA Sequence Generation Examples

The following examples use the DNA Sequence Generation (Prediction) functionality to get started with the NIM.

Both Python and Shell examples provide input DNA sequence, number of generated nucleotides (num_tokens), as well as some optional parameters. The outputs of the examples will contain JSON objects with generated DNA nucleotides: adenine (A), thymine (T), guanine (G), and cytosine (C) in the sequence field, as well as other optional fields, such as sampled probabilities per nucleotide.

For comprehensive descriptions of all available APIs and parameters, see Evo 2 NIM Endpoints. For information on assessing and ensuring model accuracy, see Benchmarking with Evo 2 NIM.

Python client example

The following is an example of how you can use a Python client to input a DNA sequence to generate its DNA nucleotides.

1. Save following Python example to a file named nim_client.py.

#!/usr/bin/env python3
import requests
import os
import json
from pathlib import Path
r = requests.post(
    url="http://localhost:8000/biology/arc/evo2/generate",
    json={
        "sequence": "ACTGACTGACTGACTG",
        "num_tokens": 8,
        "top_k": 1,
        "enable_sampled_probs": True,
    },
)
if "application/json" in r.headers.get("Content-Type", ""):
    print(r, "Saving to output.json:\n", r.text[:200], "...")
    Path("output.json").write_text(r.text)
elif "application/zip" in r.headers.get("Content-Type", ""):
    print(r, "Saving large response to data.zip")
    Path("data.zip").write_bytes(r.content)
else:
    print(r, r.headers, r.content)

2. Execute the example.

chmod +x nim_client.py
./nim_client.py

3. The example saves results to the output.json file in JSON format. You can view the file using the following command:

less output.json

Shell Client Example

1. Save the following Shell example to a file named nim_client.sh.

#!/usr/bin/env bash
set -e
URL=http://localhost:8000/biology/arc/evo2/generate
request='{
 "sequence": "ACTGACTGACTGACTG",
 "num_tokens": 8,
 "top_k": 1,
 "enable_sampled_probs": true
}'
curl -H 'Content-Type: application/json' \
     -d "$request" "$URL"

2. Execute the example. The example displays the results to the terminal in JSON format.

chmod +x nim_client.sh
./nim_client.sh