Installation#
Before running a NIM LLM container, you must authenticate with your deployment source, configure your environment, and pull the container image.
Docker Login#
To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.
Accept the Governing Terms#
Before you download a given NIM for the first time, you must accept the governing terms in the browser. Navigate to the NGC Catalog page for the NIM and click the Accept Terms button.
To find a link to the NGC Catalog page for a given NIM (for example,
meta/llama-3.1-8b-instruct), refer to the Support Matrix.
Pull the Container Image#
Once you have generated your API key and authenticated with your deployment source, you can download the NIM container image to your host machine.
Use the docker pull command to fetch the NIM container image.
To fetch a model-specific NIM container from the NGC registry, use the model’s exact image tag. For example, for Llama-3.1-8B-Instruct:
docker pull nvcr.io/nim/meta/llama-3.1-8b-instruct:2.0.1
To fetch the model-free NIM container from the NGC registry, use the following command:
docker pull nvcr.io/nim/nvidia/model-free-nim:2.0.1
Note
The exact image tag will vary depending on the container type and backend you intend to deploy. Check the Support Matrix for the model ID and version to use in the pull command for your preferred NVIDIA NIM container or model.