Installation#
Before running a NIM LLM container, you must authenticate with your deployment source, configure your environment, and pull the container image.
Docker Login#
If you created an NGC API key, authenticate with the NVIDIA Container Registry using the following command:
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.
Pull the Container Image#
Once you have generated your API key and authenticated with your deployment source, you can download the NIM container image to your host machine.
Use the docker pull command to fetch the NIM container image.
To find a link to the NGC Catalog page for a given NIM container image (for example,
meta/llama-3.1-8b-instruct), refer to the Support Matrix.
To fetch a model-specific NIM container from the NGC registry, use the model’s exact image tag. For example, for Llama-3.1-8B-Instruct:
docker pull nvcr.io/nim/meta/llama-3.1-8b-instruct:2.0.10
To fetch the model-free NIM container from the NGC registry, use the following command:
docker pull nvcr.io/nim/nvidia/model-free-nim:2.0.10
Note
The exact image tag will vary depending on the container type and backend you intend to deploy. Check the Support Matrix for the model ID and version to use in the pull command for your preferred NVIDIA NIM container or model.