Deployment

The following is used to deploy or update the NVTL API service on an existing Kubernetes cluster. One can use the following to enable HTTPS and enforce user authentication to enable secure multi-tenancy. You do not need these steps if you followed one of the Platform Setup and do not wish to enable secure multi-tenancy.

One can shutdown an already deployed NVTL service.

Copy
Copied!
            

helm delete nvtl-api

One must use the provided Helm chart to deploy NVTL services.

Copy
Copied!
            

helm fetch https://helm.ngc.nvidia.com/nvidia/tao/charts/nvtl-api-5.3.0.tgz --username='$oauthtoken' --password=<YOUR API KEY> mkdir nvtl-api && tar -zxvf nvtl-api-5.3.0.tgz -C nvtl-api

If needed, one can customize the deployment by updating the chart’s nvtl-api/values.yaml.

  • image is the location of the NVTL API container image

  • host, tlsSecret are for enabling HTTPS and enforcing user authentication, enabling secure multi-tenancy

  • corsOrigin is for enabling CORS and setting origin

  • authClientID is reserved for future NVIDIA Starfleet authentication

  • imagePullSecret is the secret name that you setup to access Nvidia’s nvcr.io registry

  • imagePullPolicy is set to Always fetch from nvcr.io instead of using locally cached image

  • storageClassName is the storage class created by your K8s Storage Provisioner. On bare-metal deployment it is nfs-client, and on AWS EKS can be standard. Not providing a value would make your deployment use your K8s cluster’s default storage class

  • storageAccessMode is set to ReadWriteMany to reuse allocated storage between deployments, or ReadWriteOnce to create a new storage at every deployment

  • storageSize is ignored by many Storage Provisioners. But here would be where to set your shared storage size

  • backend is the platform used for training jobs. Defaults to local-k8s

  • maxNumGpuPerNode is the number of GPU assigned to each job. Multi-node training is not supported, you are limited to the number of GPUs within a cluster node

  • telemetryOptOut can be set if you want to opt-out from NVIDIA to collect anonymous usage metrics

Example for creating a tlsSecret:

Copy
Copied!
            

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=ec2-34-221-205-157.us-west-2.compute.amazonaws.com/O=ec2-34-221-205-157.us-west-2.compute.amazonaws.com" --addext "subjectAltName = DNS:ec2-34-221-205-157.us-west-2.compute.amazonaws.com" kubectl create secret tls tls-secret --key tls.key --cert tls.crt --namespace default

Then deploy the API service.

Copy
Copied!
            

helm install nvtl-api nvtl-api/ --namespace default

One can validate the deployment by looking for the Ready and Completed states.

Copy
Copied!
            

kubectl get pods -n default

To debug a deployment. look for events toward the bottom of the following command.

Copy
Copied!
            

kubectl describe pods nvtl-api -n default

Common issues are:

  • GPU Operator or Storage Provisioner pods not in Ready or Completed states

  • Missing or invalid imagepullsecret

Previous Setup
Next Nvidia Transfer Learning REST API
© Copyright 2024, NVIDIA. Last updated on Mar 22, 2024.