Installation#
Before running a NIM LLM and VLM container, configure your environment and pull the container image. Most current public NIM images support keyless access from NGC, so Docker login is optional unless your image, tag, or model artifact requires credentials.
Docker Login#
If you created an NGC API key, authenticate with the NVIDIA Container Registry using the following command:
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.
Pull the Container Image#
Use docker pull to fetch the NIM container image. For eligible public-catalog
NIMs, you can pull the image without Docker login. Authenticate first only when
you use a Production Branch (PB) NIM, a NIM released prior to
NIM LLM and VLM version 2.0.10, or a private or gated NIM image or
model artifact.
To find a link to the NGC Catalog page for a given NIM container image (for example,
meta/llama-3.1-8b-instruct), refer to the Support Matrix.
To fetch a model-specific NIM container from the NGC registry, use the model’s exact image tag. For example, for Llama-3.1-8B-Instruct:
docker pull nvcr.io/nim/meta/llama-3.1-8b-instruct:2.0.12
To fetch the model-free NIM container from the NGC registry, use the following command:
docker pull nvcr.io/nim/nvidia/model-free-nim:2.0.12
Note
The exact image tag will vary depending on the container type and backend you intend to deploy. Check the Support Matrix for the model ID and version to use in the pull command for your preferred NVIDIA NIM container or model.