List Models

List all available models through the NIM Proxy microservice by running a GET API call.

Types of Models Auto-detected by NIM Proxy

• Fine-tuned models uploaded by NeMo Customizer.

• NIM deployed with the NeMo Deployment Management microservice.

• NIM deployed using Helm with the following label added to the spec object of the NIMService custom resource:

  spec:
     labels:
        app.nvidia.com/nim-type: inference
  

---

Prerequisites

Before you can list models, make sure that you have:

• Access to the NIM Proxy microservice through the base URL where the service is deployed, either as part of the platform or standalone. Store the base URL in an environment variable NIM_PROXY_BASE_URL.

---

Options

You can list models in the following ways.

API

1. Perform a GET request to the /v1/models endpoint.

   cURL

   Use one of the following cURL commands. For more details on the request body, see the NIM Proxy API reference.

   curl -X GET \
     "${NIM_PROXY_BASE_URL}/v1/models" \
     -H 'accept: application/json' | jq
   

2. Review the response.

   Example Response

   {
     "data": [
       {
         "id": "llama2-7b",
         "object": "model",
         "created": 1677610602,
         "owned_by": "meta"
       },
       {
         "id": "gpt-3.5-turbo",
         "object": "model",
         "created": 1677610602,
         "owned_by": "openai"
       }
     ],
     "object": "list"
   }