Prerequisites

This documentation describes the prerequisites for the Semantic Split C-RADIO NIM.

Hardware Support

The Early Access release is specifically fine-tuned for L40S. In particular, NVEnc enabled hardware is required. Please contact us for inquiries on how the NIM could be extended to support more hardware configurations.

Software

• With a Docker-supported operating system, install Docker - minimum version: 23.0.1

• Install NVIDIA Drivers - minimum version: 535. You can use the NVIDIA driver release 535.86 (or later R535), or 545.23 (or later R545) which are supported on L40S DataCenter GPUs.

• Install and configure the NVIDIA Container Toolkit - minimum version: 1.13.5

• Verify your container runtime supports NVIDIA GPUs by running the following:

  docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

For more information on enumerating multi-GPU systems, please see the NVIDIA Container Toolkit’s GPU Enumeration Docs.

NGC

You must have an authenticated NGC (NVIDIA GPU Cloud) account with access to both the respective NIM and the model checkpoints. Use the following procedure to log in to NGC, and set the NGC_API_KEY environment variable, so that you can pull images.

• Create an account on NGC.

• Generate an API Key. The following steps require your NGC API key.

• Authenticate local Docker with NGC by running the following code. For more details, see the NGC authentication documentation.

  docker login nvcr.io
  Username: $oauthtoken
  Password: <NGC API key>

• Set the NGC_API_KEY environment variable in your shell.

  export NGC_API_KEY=<NGC API key>

Supported Formats