API#

NGC Domain (required for some APIs): https://api.ngc.nvidia.com

OpenAPI Specification#

This page is a brief overview of using the NVCF API and does not cover all endpoints.

Please refer to the OpenAPI Spec for the latest API information. The quickest way to begin using NVCF APIs is via the Postman Collection.

The NVCF API is divided into the following sets of APIs:

APIs	Usage
Function Invocation	Execution of a function that runs on a worker node. Usually an inference call.
Asset Management	Used to manage large files for uploading for a requested and downloading results of a function.
Cluster Groups & GPUs	Defines endpoints to list Cluster Groups and GPUs as targets for function deployment.
Queue Details	Used to view information about your environment such as queues & GPUs.
Function Management	The creation, modification and deletion of functions
Function Deployment	Endpoints for creating and managing function deployments.

API Versioning

All API endpoints include versioning in the path prefix.

/v2/nvcf

Authorization#

The NVCF API supports NGC API key-based authorization for calling the API directly, or indirectly via the NGC CLI and NGC SDK.

The generated NGC Personal API Key will also be used for pushing and pulling containers, models, resources and helm charts to the NGC Private Registry to use during function creation.

Generate an NGC Personal API Key#

The API Key can be generated via your account in the Personal Keys Page.

Warning

You must have the Cloud Functions and, if using, Cloud Tasks roles associated to your user at the Organization level not the Team level for the generated key to work correctly. See here for more information.

It’s recommended that the API Key that you generate includes both NGC Catalog and Private Registry services for seamless usage with the NGC CLI.

Warning

API Key scopes are static. This means if the key is lost, it must be destroyed and recreated.

For more information about NGC API Key management see the NGC API Key documentation.

API Key Usage

The API Key is passed in the Authorization header.

Authorization: Bearer $API_KEY

API Key Scopes and Domains#

Note

There are multiple API key types within NGC. We strongly recommend using the NGC Personal API Key for complete NVCF API compatibility.

Required domain names are documented below and are also pre-filled within our Postman Collection.

Our OpenAPI Spec also describes the scopes required for each endpoint.

Scope Name	Domain Name	API Category
update_function	https://api.ngc.nvidia.com	Function Management
register_function	https://api.ngc.nvidia.com	Function Management
queue_details	https://api.nvcf.nvidia.com	Queue Details
list_functions	https://api.nvcf.nvidia.com	Function Management
list_cluster_groups	https://api.ngc.nvidia.com	Cluster Groups and GPUs
invoke_function	https://api.nvcf.nvidia.com	Function Invocation and Asset Management
deploy_function	https://api.ngc.nvidia.com	Function Deployment
delete_function	https://api.ngc.nvidia.com	Function Management

NVCF API Status Codes#

Please refer to the OpenAPI Docs for other possible status code failure reasons in cases where they are not generated from your inference container.

For Function States, see Function Lifecycle.

Note

To easily differentiate between errors originating from within NVCF’s API or control plane, and your own inference container, determine if the type field includes inference-service (indicating the error is from your inference container)