Quickstart - NVIDIA Docs

This guide gives an end to end workflow of how to create and deploy a container-based function on Cloud Functions, using the fastapi_echo_sample container. This is a minimal server that’s been configured to work with Cloud Functions, echoing whatever is sent to it’s inference endpoint.

Before beginning, ensure that you have created an NGC Personal API Key and authenticated with the NGC Docker Registry.

Clone, Build and Push the Docker Image to NGC Private Registry

First clone and build the docker image.

Copy
Copied!

            
            > git clone https://github.com/NVIDIA/nv-cloud-function-helpers.git
> cd nv-cloud-function-helpers/examples/fastapi_echo_sample
> docker build . -t fastapi_echo_sample

Now tag and push the docker image to NGC Private Registry.

Copy
Copied!

            
            > docker tag fastapi_echo_sample:latest nvcr.io/$ORG_NAME/fastapi_echo_sample:latest
> docker push nvcr.io/$ORG_NAME/fastapi_echo_sample:latest

Once this finishes, you’ll now be able to see the new container in the NGC Private Registry Containers Page and it will be available for use in function creation.

Create & Deploy the Function Using the Cloud Functions UI

Navigate to the Cloud Functions UI and choose “Create Function”, then “Custom Container”. Enter in the details for the fastapi_echo_sample. In this container, the inference endpoint is /echo, the health endpoint is /health and it exposes these on the default 8000 port. No other configuration changes are needed.

Once created, the function will show up in the Functions List Page with status INACTIVE because it is not yet deployed. By default, a function ID and function version ID is generated every time you create a new function, or new function version.

Deploy the function by clicking the kebab menu on the right of the function row.

Create & Deploy the Function Using the Cloud Functions API

Ensure you have an API key created, see Generate an NGC Personal API Key.
Find your NGC organization name within the NGC Organization Profile Page. This is not the Display Name. For example: qdrlnbkss123.
Create the function via API by running the following curl with an $API_KEY and your $ORG_NAME.

Copy
Copied!

            
             curl --location 'https://api.ngc.nvidia.com/v2/nvcf/functions' \
 --header 'Content-Type: application/json' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer $API_KEY' \
 --data '{
"name": "my-echo-function",
"inferenceUrl": "/echo",
"healthUri": "/health",
"inferencePort": 8000,
"containerImage": "nvcr.io/$ORG_NAME/fastapi_echo_sample:latest"
}'

It will return a function ID listed as id and a function version ID listed as versionId.

List functions you’ve created via API using the following curl:

Copy
Copied!

            
             curl --location 'https://api.ngc.nvidia.com/v2/nvcf/functions?visibility=private' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer $API_KEY' \

Deploy the function via API using the following curl:

Copy
Copied!

            
             curl --location 'https://api.ngc.nvidia.com/v2/nvcf/deployments/functions/$FUNCTION_ID/versions/$FUNCTION_VERSION_ID' \
 --header 'Content-Type: application/json' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer $API_KEY' \
 --data '{
"deploymentSpecifications": [
{
"backend": "GFN",
"gpu": "L40G",
"maxInstances": "1",
"minInstances": "1"
}
]
}'

Refer to the OpenAPI Specification for further API documentation.

Create & Deploy the Function Using the NGC CLI

Ensure you have an API key created, see Generate an NGC Personal API Key.
Ensure you have the NGC CLI configured.
Run the following command to create the function, it will return a function ID and function version ID.

Copy
Copied!

            
             ngc cf function create --health-uri /health --inference-port 8000 --container-image nvcr.io/$ORG_NAME/fastapi_echo_sample:latest --name my-echo-function --inference-url /echo

List functions you’ve created using the following command:

Copy
Copied!

            
             ngc cf function list --access-filter=private

Deploy the function using the following command:

Copy
Copied!

            
             ngc cf function deploy create --deployment-specification GFN:L40:gl40_1.br20_2xlarge:1:1 $FUNCTION_ID:$FUNCTION_VERSION_ID

See NGC CLI Documentation. for further commands.

Invoke the Function

Once your function is deployed, invoke it’s inference endpoint with the following curl:

Copy
Copied!

            
             curl --location 'https://api.nvcf.nvidia.com/v2/nvcf/pexec/functions/$FUNCTION_ID' \
 --header 'Content-Type: application/json' \
 --header 'Authorization: Bearer $API_KEY' \
 --data '{
"message": "hello world"
}'