Quickstart for Boltz-2
This Quickstart Guide walks you through authenticating with NGC, starting the Boltz-2 NIM container, and running your first prediction request. For comprehensive system requirements and verified configurations, refer to the Support Matrix.
Prerequisites
Before you begin, make sure you have the following:
-
Refer to the Support Matrix to make sure that you have the supported hardware and software stack.
-
An NGC personal API key. The NIM microservice uses the API key to download models from NVIDIA NGC. Refer to Generating a Personal API Key in the NVIDIA NGC User Guide for more information.
-
To run the following examples you must have
curl,jq, and the Pythonrequestsmodule installed.When you create an NGC API personal key, select at least the option for NGC Catalog from the Services Included menu. You can specify more services to use the key for additional purposes.
Installing curl, jq, and the Python requests Module
Install each of the following tools:
-
Install
curlto send requests from the command line.On Ubuntu and other Debian-based Linux distributions, install it with the following command, which could require
sudoaccess: -
Install
jqto process and reformat responses from the NIM so they are easier to read.Install it using the following command, which may require
sudoaccess: -
Install the Python
requestsmodule, which facilitates HTTP requests made in many of the examples for this NIM.Install it using
pip:
NGC Authentication
The following steps generate and export the NGC API key that the NIM uses to download models.
Generate an API Key
To access NGC resources, you need an NGC API key. You can generate a key here: API Keys.
When creating an NGC API Personal key, ensure that “NGC Catalog” is selected from the “Services Included” dropdown. You can include more services if you reuse this key for other purposes.
Personal keys allow you to configure an expiration date, revoke, or delete the key using an action button, and rotate the key as needed. For more information about key types, refer to the NGC User Guide.
Export the API Key
Pass the value of the API key to the docker run command in the next section as the NGC_API_KEY environment variable to download the appropriate models and resources when starting the NIM.
If you are not familiar with how to create the NGC_API_KEY environment variable, export it in your terminal:
Run one of the following commands to make the key available at startup:
Other, more secure options include saving the value in a file, so that you can retrieve with cat $NGC_API_KEY_FILE, or using a password manager.
Docker Login to NGC
To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:
Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a username and password.
Quickstart Steps
To start the Boltz-2 NIM and run your first prediction, complete the following steps:
-
Confirm you have logged in to Docker and set your
NGC_API_KEYenvironment variable, as detailed in the preceding section. -
Create a NIM Cache.
If one already exists, you may use the existing one.
-
Start the NIM container:
- Mounting
/opt/nim/outputkeeps prediction artifacts (confidence JSONs, PAE/PDE.npzfiles) separate from the model cache. Refer to Volumes. To serve fine-tuned weights, refer to Custom Checkpoints. - The
--runtime=nvidiaflag is the legacy approach for GPU access and might require additional Docker daemon configuration. For Docker 19.03 or later, you can use the--gpusflag instead, which works without additional configuration.
Use all GPUs:
Or specify one GPU device:
The preceding command starts the container with the default profile, using all available system GPUs.
This requires localizing up to 30 GB of data to disk. If you have the model cached, the version in the NIM cache specified at
LOCAL_NIM_CACHEis used. - Mounting
-
(Optional) Confirm the service is ready to respond to inference requests:
Example Output
-
Send an inference request:
Curl
Python
The description of all API parameters is in the Input parameters section of the Inference page.
Runtime Parameters for the Container
The following table lists the runtime flags used with the docker run command.
Caching Models
The first time you start the container, the microservice downloads the model from NGC. You can avoid this download step on future runs by caching the model locally.
Stopping the Container
The following commands stop the container by stopping and removing the running docker container.
Running on Slurm With Enroot
This section describes how to run the Boltz2 NIM on a Slurm-based HPC cluster using Enroot to run the NGC container.
NGC_API_KEY: You must export your NGC API key before running the Slurm script so that Enroot can pull the image from NGC. Then run the script (for example, bash boltz2_srun.sh).
Before you start, you must complete the following prerequisites:
-
Install Enroot version 3.4.1 or newer on the cluster.
-
Export your NGC API key in your environment so that Enroot can pull the image from NGC:
To run the Boltz-2 NIM on Slurm with Enroot, complete the following steps:
-
Check that Enroot is available.
On the login node, verify Enroot is installed:
If Enroot is not installed, contact your cluster administrator. Create the Enroot config directory if it does not exist:
-
Create and add NGC credentials.
Create the credentials file so Enroot can pull images from NGC. Replace
YOUR_NGC_API_KEYwith your NGC API key: -
Configure and run the Slurm job script.
Copy the script below to a file, such as
boltz2_srun.sh, and set the variables at the top for your cluster and paths:- ACCOUNT: Slurm account name
- PARTITION: Slurm partition (for example,
interactiveorgpu) - ROOT_DIR: Your working directory on the cluster (for example, Lustre or NFS path)
- NGC_API_KEY: Must be exported in your environment before running the script (for example,
export NGC_API_KEY=<your_key>)
Use a shared filesystem path for
ROOT_DIRandCACHE_PATHso that the SQSH image and cache are visible from the node where the job runs. Save the following asboltz2_srun.shand run it withbash boltz2_srun.sh: -
Create an SSH tunnel from your machine or the login node, after the container is running on a compute node:
-
Send an inference request.
In another terminal, with the tunnel open, run the following command:
If the request succeeds, you will get a JSON response containing the prediction result.
Next Steps
When you are ready, continue with the following:
- Refer to Inferencing with the NIM for the full API reference and usage examples.
- Refer to Optimization for settings related to optimizing performance of the NIM on your hardware and software stack.