Configure The NIM at Runtime | NVIDIA NIM for Semantic Split C-RADIO

GPU Selection

You can use the --gpus all argument to docker run in homogeneous environments with one or more of the same GPU. In heterogeneous environments with a combination of GPUs, such as L40S and a GeForce display GPU, you should only run workloads on L40S GPUs. Use the following option to identify specific GPUs:

The environment variable CUDA_VISIBLE_DEVICES. For example: -e CUDA_VISIBLE_DEVICES=1.

Use the nvidia-smi -L command to get the device ID(s) to use as input(s). This command should return information similar to the following:

GPU 0: NVIDIA GeForce RTX 3080
GPU 1: L40S

For more information, see NVIDIA Container Toolkit documentation.

Environment Variables

You can pass the following environment variables into a NIM by adding the -e flag to the docker run command.

ENV	Required?	Default	Notes
NGC_API_KEY	Yes	None	Your NGC API key with read access to the model registry for the model profile you are using.
NIM_LOG_LEVEL	Optional	None	You can pass `DEBUG` value to enable extra logging.
NIM_T RITON_LOG_VERBOSE	Optional	None	You can pass verbosity level [1,2,3,4,5] to enable extra logging from the triton server backend.