Getting Started
Prerequisites
To ensure that you have the supported hardware and software stack, check the Support Matrix.
NGC Authentication
Generate an API Key
An NGC API key is required to access NGC resources. You can generate a key at https://org.ngc.nvidia.com/setup/api-keys.
When creating an NGC API Personal key, ensure that at least NGC Catalog is selected from the Services Included dropdown. You can include more services if this key is to be reused for other purposes.
Personal keys allow you to configure an expiration date, revoke or delete the key using an action button, and rotate the key as needed. For more information about key types, refer to NGC API Keys in the NGC User Guide.
Export the NGC API Key
Pass the value of the API key to the docker run command in the next section as the NGC_API_KEY environment variable to download the appropriate models and resources when starting the NIM.
If you are not familiar with how to create the NGC_API_KEY environment variable, the simplest way is to export it in your terminal:
Run one of the following commands to make the key available at startup:
Other, more secure options include saving the value in a file, so that you can retrieve with cat $NGC_API_KEY_FILE, or using a password manager.
Docker Login to NGC
To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:
On success, Docker prints Login Succeeded.
Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.
Launching the NIM Container
The following command launches the Relighting container with the gRPC service. (For a list of parameters, see Runtime Parameters for the Container.)
Run the NIM launch command. The NIM selects the model profile that matches your GPU (refer to Model Manifest Profiles to pin one). The command mounts a model cache directory from the host, so later starts reuse the downloaded models:
The first start downloads the models into LOCAL_NIM_CACHE, and later starts reuse them. To remove the cached models, refer to Model Caching.
If the NIM launch is successful, the log ends with lines similar to the following. Ports 9000 and 9001 are Triton’s ports inside the container; the NIM serves inference on gRPC port 8001.
If the NIM fails to start, the container exits, and the last lines of the log name the error.
By default, the Relighting gRPC service is hosted on port 8001. You must use this port for inferencing requests. The port is configurable with the NIM_GRPC_API_PORT environment variable.
Model Manifest Profiles
The NIM selects the model profile for your GPU at startup. To list the profiles in the image you pulled, run the list-model-profiles utility:
The output lists the profiles that are compatible with your GPU and the ones that are not, with the compute capability of each. To pin a profile, add -e NIM_MODEL_PROFILE={profile_id} to the launch command, with an ID that the output lists as compatible.
With a profile ID for another GPU architecture, the NIM does not find the engines for your GPU: it logs Model validation failed after 120s and exits.
Environment Variables
The following table describes the environment variables that can be passed into a NIM as a -e argument added to a docker run command:
Runtime Parameters for the Container
Stopping the Container
The launch commands on this page do not use --rm, so stop the container and then
remove it: