Quickstart Guide for Evo 2 NIM

Start NIM

1. Export the NGC_API_KEY variable.

export NGC_API_KEY=<your personal NGC key>

2. The NIM container downloads models automatically. To save time and bandwidth, provide a local cache directory so the NIM can reuse models it has already downloaded. Run the following command to set up the cache directory.

export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
sudo chmod 0777 -R "$LOCAL_NIM_CACHE"

3. Run the NIM container with the following commands.

docker run -it \
    --runtime=nvidia \
    --gpus='"device=0,1"' \
    -p 8000:8000 \
    -e NGC_API_KEY \
    -v "$LOCAL_NIM_CACHE":/opt/nim/.cache \
    nvcr.io/nim/arc/evo2:2

This command starts the NIM container and exposes port 8000. The container defaults to the 40B model.

4. Open a new terminal, and leave the terminal with the service running. In the new terminal, wait until the health endpoint returns {"status":"ready"} before proceeding. This can take a couple of minutes. Query the health endpoint with the following command.

curl http://localhost:8000/v1/health/ready

DNA Sequence Generation Examples

The following examples use the DNA Sequence Generation (Prediction) functionality to get started with the NIM.

Both Python and Shell examples provide input DNA sequence, number of generated nucleotides (num_tokens), as well as some optional parameters. The outputs of the examples will contain JSON objects with generated DNA nucleotides: adenine (A), thymine (T), guanine (G), and cytosine (C) in the sequence field, as well as other optional fields, such as sampled probabilities per nucleotide.

For descriptions of the APIs and parameters, refer to Evo 2 NIM Endpoints. For information on assessing model accuracy, refer to Benchmarking with Evo 2 NIM.

Python Client Example

The following is an example of how you can use a Python client to input a DNA sequence to generate its DNA nucleotides.

1. Save the following Python example to a file named nim_client.py.

#!/usr/bin/env python3
import requests
import os
import json
from pathlib import Path
r = requests.post(
    url="http://localhost:8000/biology/arc/evo2/generate",
    json={
        "sequence": "ACTGACTGACTGACTG",
        "num_tokens": 8,
        "top_k": 1,
        "enable_sampled_probs": True,
    },
)
if "application/json" in r.headers.get("Content-Type", ""):
    print(r, "Saving to output.json:\n", r.text[:200], "...")
    Path("output.json").write_text(r.text)
elif "application/zip" in r.headers.get("Content-Type", ""):
    print(r, "Saving large response to data.zip")
    Path("data.zip").write_bytes(r.content)
else:
    print(r, r.headers, r.content)

2. Execute the example.

chmod +x nim_client.py
./nim_client.py

3. The example saves results to the output.json file in JSON format. You can view the file using the following command:

less output.json

Shell Client Example

1. Save the following Shell example to a file named nim_client.sh.

#!/usr/bin/env bash
set -e
URL=http://localhost:8000/biology/arc/evo2/generate
request='{
 "sequence": "ACTGACTGACTGACTG",
 "num_tokens": 8,
 "top_k": 1,
 "enable_sampled_probs": true
}'
curl -H 'Content-Type: application/json' \
     -d "$request" "$URL"

2. Execute the example. The example displays the results to the terminal in JSON format.

chmod +x nim_client.sh
./nim_client.sh