Is this page helpful?

Prerequisites#

This page outlines the hardware and software requirements for running Alpamayo1.5 NIM.

Hardware Requirements#

  • GPU: Alpamayo1.5 NIM currently supports one GPU per container. Supported GPUs and profile constraints are listed in Support Matrix. Validated GPUs have published precision and K caps. Other NVIDIA GPUs can use the generic compatibility path when they have compute capability 8.0 or newer and sufficient total and currently free memory. Quantized profiles require at least 20 GB; BF16 requires at least 30 GB; FP8 additionally requires compute capability 8.9 or newer.

  • CPU: x86_64 architecture is supported.

  • CPU memory: At least 64 GB of RAM is recommended for a single default Alpamayo1.5 NIM container. Use additional host memory when running multiple containers on the same host or when configuring broad high-K serving.

  • Disk space: At least 100 GB of disk space is recommended for the default container image, model artifacts, and persistent cache. Use 150 GB or more when using BYOC checkpoints or creating quantized exports at startup.

Software Requirements#

  • Operating system: Any Linux distribution that is supported by NVIDIA Container Toolkit and has glibc version 2.35 or later installed.

  • NVIDIA GPU driver: Version 580 or later.

  • CUDA drivers: Follow the CUDA installation guide to install CUDA drivers. The CUDA toolkit itself is not required on the host because required CUDA libraries are packaged in the NIM container.

  • Docker: Version 23.0.1 or later. Follow the Docker Engine installation guide.

  • NVIDIA Container Toolkit: Version 1.16.2 or later. Follow the NVIDIA Container Toolkit installation guide.

NGC Access#

You need an NGC account and an API key with access to the NGC Catalog to pull the container and download its model artifacts. Create and export the key as described in Export the API Key, and authenticate Docker before pulling the image:

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

The key is also passed to the container on its first connected startup. The Quickstart Guide shows the complete pull-and-run flow. For a deployment that must run without network access, first prepare the cache by following Air-Gapped Deployment.

Configuration and Verification#

After installing the NVIDIA Container Toolkit, configure Docker by following the NVIDIA Container Toolkit documentation.

To verify that Docker can access the GPU, run:

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

The command should list the available NVIDIA GPUs and driver version.