Installation#
Before running a NIM LLM container, you must authenticate with your deployment source, configure your environment, and pull the container image.
Docker Login#
To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.
Pull the Container Image#
Once you have generated your API key and authenticated with your deployment source, you can download the NIM container image to your host machine.
Use the docker pull command to fetch the NIM container image.
To fetch a model-specific NIM container from the NGC registry, use the model’s exact image tag. For example, for Llama-3.1-8B-Instruct:
docker pull nvcr.io/nim/meta/llama-3.1-8b-instruct:2.0.1
To fetch the model-free NIM container from the NGC registry, use the following command:
docker pull nvcr.io/nim/nvidia/model-free-nim:2.0.1
Note
The exact image tag will vary depending on the container type and backend you intend to deploy. Check the Support Matrix for the model ID to use in the pull command for your preferred NVIDIA NIM container or model.