Skip to navigation

Configuring the NIM at Runtime

View as Markdown

The AlphaFold2-Multimer NIM allows a number of customizations at runtime that allow for:

  • Tuning the performance of the NIM.
  • Taking advantage of your unique hardware.
  • Modifying runtime variables (such as the location of the NIM cache).

The AlphaFold2-Multimer NIM runtime options follow. For the first-run procedure, refer to the Quickstart Guide.

Start the AlphaFold2-Multimer NIM

First, review how to start the NIM:

export LOCAL_NIM_CACHE=~/.cache/nim
docker run -it --rm --name alphafold2-multimer --runtime=nvidia \
    -e CUDA_VISIBLE_DEVICES=0 \
    -e NGC_CLI_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache \
    -p 8000:8000 \
    nvcr.io/nim/deepmind/alphafold2-multimer:1.0.0

Remember:

  • -p defines the port used to query the NIM.
  • -e options define environment variables, which are passed into the NIM container at runtime.
  • --rm removes the container when it exits, for example after Ctrl+C.
  • -it allows interacting with the container terminal directly in the CLI, for example with Ctrl+C.

If the docker run command fails due to file permission errors after downloading the AlphaFold2 model, run the following command: sudo chmod -R 777 $LOCAL_NIM_CACHE. Afterward, rerun the NIM using the docker run command.

Change the NIM Cache Location

Running the NIM container automatically downloads the AlphaFold2 model to a default path in the container filesystem: /opt/nim/.cache. The location of the NIM model cache can be changed on both the local filesystem and within the NIM. This is useful when a larger drive is needed to store the MSA databases or when you want a different workspace inside the container. The MSA databases account for a large fraction of the AlphaFold2 model size (about 613 GB).

To change the NIM cache location on the host filesystem, change the source mount location in the NIM startup command. That location is the LOCAL_NIM_CACHE environment variable:

export LOCAL_NIM_CACHE=/mount/largedisk/nim/.cache
docker run -it --rm --name alphafold2-multimer --runtime=nvidia \
    -e CUDA_VISIBLE_DEVICES=0 \
    -e NGC_CLI_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache \
    -p 8000:8000 \
    nvcr.io/nim/deepmind/alphafold2-multimer:1.0.0

This changes only the location outside the container, on the host running the container. To change the location of the cache mounted inside the container, you must:

  1. Mount the new destination path: -v $LOCAL_NIM_CACHE:$NIM_CACHE_PATH.
  2. Provide the new path in the NIM_CACHE_PATH environment variable.

This example moves the cache within the container to /workdir/cache/af2/:

export LOCAL_NIM_CACHE=/mount/largedisk/nim/.cache
docker run -it --rm --name alphafold2-multimer --runtime=nvidia \
    -e CUDA_VISIBLE_DEVICES=0 \
    -e NGC_CLI_API_KEY \
    -e NIM_CACHE_PATH=/workdir/cache/af2 \ ## Note: we must set this environment variable...
    -v $LOCAL_NIM_CACHE:/workdir/cache/af2 \ ## and provide the destination mount.
    -p 8000:8000 \
    nvcr.io/nim/deepmind/alphafold2-multimer:1.0.0

Run an Air-Gapped Deployment

To download, detect, and verify the cached model, the NIM needs internet connectivity to NGC. That check cannot run when the NIM is offline or air-gapped.

To work around these constraints, configure the NIM to ignore model download and validation errors on startup, and mount a locally downloaded AF2 model to the offline NIM server:

  1. Download the model to a local disk that can be mounted on the offline host.
  2. Mount the cached model on the offline host when you run the NIM.
  3. Set NIM_IGNORE_MODEL_DOWNLOAD_FAIL=true so that the container attempts model download and detection, fails without terminating, and uses the mounted model.

To download the model to your local disk, run the following command:

ngc registry model download-version nim/deepmind/alphafold2-data:1.1.0

Start the NIM container and mount the cached model from the path specified in $LOCAL_NIM_CACHE:

docker run -it --rm --name alphafold2-multimer --runtime=nvidia \
    -e CUDA_VISIBLE_DEVICES=0 \
    -e NGC_CLI_API_KEY \
    -e NIM_IGNORE_MODEL_DOWNLOAD_FAIL=true \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache \
    -p 8000:8000 \
    nvcr.io/nim/deepmind/alphafold2-multimer:1.0.0

Use an Alternative Port

If you have other HTTP or gRPC servers running (for example, other NIMs), you might need to make port 8000 available for those servers by using another port for your NIM. To do so:

  1. Change the exposed port, for example -p $HOST_PORT:$CONTAINER_PORT.
  2. Set the NIM_HTTP_API_PORT environment variable to the new container port.

This example runs the NIM on port 7979. It does not change the cache path inside the container.

export LOCAL_NIM_CACHE=/mount/largedisk/nim/.cache
docker run -it --rm --name alphafold2-multimer --runtime=nvidia \
    -e CUDA_VISIBLE_DEVICES=0 \
    -e NGC_CLI_API_KEY \
    -e NIM_HTTP_API_PORT=7979 \ ## We must set the NIM_HTTP_API_PORT environment variable...
    -e NIM_CACHE_PATH=/workdir/cache/af2 \
    -v $LOCAL_NIM_CACHE:/workdir/cache/af2 \
    -p 7979:7979 \ ## as well as forward the port to host.
    nvcr.io/nim/deepmind/alphafold2-multimer:1.0.0

Set Parallel MSA Runners and Threads

The AlphaFold2-Multimer NIM can tune the MSA process for the number of available CPU cores, the number of available GPUs, and the speed of the SSD.

Do not run the AlphaFold2-Multimer NIM on a machine that uses a hard disk drive for storage. A hard disk drive decreases NIM performance.

There are two dimensions of tunable parallelism for MSA within the AlphaFold2-Multimer NIM, set by two environment variables:

  1. NIM_PARALLEL_MSA_RUNNERS: Controls the coarse-grained parallelism of the MSA process.
  2. NIM_PARALLEL_THREADS_PER_MSA: Controls the number of threads used by each MSA process.

The product of NIM_PARALLEL_MSA_RUNNERS and NIM_PARALLEL_THREADS_PER_MSA should be ≥ the number of CPU cores available to the NIM.

For example, on a machine with 48 CPU cores, an appropriate setting would be:

NIM_PARALLEL_MSA_RUNNERS=4
NIM_PARALLEL_THREADS_PER_MSA=12

This allocates 4 runners, each with 12 threads.

Do not use more than 5 process runners. The NIM performs well when the number of threads per MSA is a factor of the number of CPU threads. The default values of NIM_PARALLEL_MSA_RUNNERS=3 and NIM_PARALLEL_THREADS_PER_MSA=8 work for most machines.

To modify these variables at runtime, pass them as environment variables in the NIM startup command. For example, on a 64-core machine, when running many MSA databases, set NIM_PARALLEL_MSA_RUNNERS to 6 and NIM_PARALLEL_THREADS_PER_MSA to 10:

docker run -it --rm --name alphafold2-multimer --runtime=nvidia \
    -e CUDA_VISIBLE_DEVICES=0 \
    -e NGC_CLI_API_KEY \
    -e NIM_PARALLEL_MSA_RUNNERS=6 \ ## We set the number of parallel runners to 6, perhaps because we have used many MSA databases...
    -e NIM_PARALLEL_THREADS_PER_MSA=10 \ ## and we set the number of threads per MSA to 12, to distribute alignment workloads...
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache \
    -p 8000:8000 \
    nvcr.io/nim/deepmind/alphafold2-multimer:1.0.0