Create Dedicated Endpoint from NVIDIA NIM

For enhanced performance and seamless compatibility, NVIDIA-optimized models using NIM container registry are also available on DGX Cloud Lepton.

Prerequisites

The NVIDIA-optimized models require an NVIDIA account with access to the NIM container registry.

You need to have an NVIDIA account with access to the NIM container registry, and configure the registry auth key on DGX Cloud Lepton.

Refer to this guide for more details.

Besides the registry auth key, you also need to have an NGC API key. Navigate to the NGC API key creation page, click on Generate Personal Key.

In the Service Included field, select Public API Endpoints.

Then you can store the NGC API key on DGX Cloud Lepton as a secret.

Navigate to the create endpoint page on the dashboard.

In the NIM configuration section, you can specify the container image, registry auth key, and NGC API key.

Image: The container image to use for the endpoint, basically starts with nim/ prefix. You can select from the preset list or input your own image URL.
Registry auth: You can select one of the existing registry auth you created in your workspace or create a new one.
NGC API key: You can select one of the existing NGC API key you saved in your workspace as a secret or create a new one.

For other endpoint configurations, you can refer to this guide.