TAO Toolkit v5.2.0

The TAO Toolkit API exposes dataset and model endpoints for setting up and running actions.

If using Python requests for making API calls, it is required to use Python >= 3.10 to avoid the upload of large files issue.

User authentication is based on NGC API KEY.

Within the cluster, one can use the following command to know the node-port for the ingress controller. The node port would be the number after the 443 cluster port 443:<NodePort>.


kubectl get services ingress-nginx-controller

the IP address would be the external IP of one of the cluster nodes.



hostname -i

A curl command to do user authentication would look like the following, where is one of the node’s external IP, 30108 is the NodePort for the ingress controller and the NGC API KEY is zZYtczM5amdtdDcwNjk0cnA2bGU2bXQ3bnQ6NmQ4NjNhMDItMTdmZS00Y2QxLWI2ZjktNmE5M2YxZTc0OGyM. This will return the user ID and token.


curl -s -X POST -d '{"ngc_api_key": "'"zZYtczM5amdtdDcwNjk0cnA2bGU2bXQ3bnQ6NmQ4NjNhMDItMTdmZS00Y2QxLWI2ZjktNmE5M2YxZTc0OGyM"'"}'

The following is a curl example of an API call with user authentication.


curl \ -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJsb2dnZWRJbkFzIjoiYWRtaW4iLCJpYXQiOjE0MjI3Nzk2Mz"

The TAO Toolkit API service includes methods for dealing with the content of experimental workspaces, such as user datasets and models. It also includes methods for executing TAO actions applicable to data and specifications stored in such experimental workspaces.

A basic scenario would be for a user to first create a Dataset for a specific network type, create a Model experiment that is pointing to the dataset and a pretrained model, and customize specs before executing network-related actions.

/api/v1/dataset | /api/v1/model

* List datasets for user
* Retrieve a new dataset
* Delete a dataset
* Create a dataset
* Update dataset metadata
* Update dataset metadata partially
* Dataset upload
* Retrieve default action specs
* Retrieve dataset action specs
* Save dataset action specs
* Update dataset action specs
* Run dataset actions
* List dataset jobs
* Retrieve dataset job
* Cancel dataset job * Delete a dataset job * List files of a dataset job * Download selective files of a dataset job * Download dataset action job

* List models for user
* Retrieve a model
* Delete a model
* Create a new model
* Update model metadata
* Update model metadata partiall
* Retrieve default action specs
* Retrieve model action specs
* Save model action specs
* Update current model specs
* Run model actions
* List model actions
* Retrieve model job
* Early stop / Cancel model actions
* Delete a model job
* Download model action job
* List files of a model job
* Download selective files of a model job
* Resume training

See the TAO Toolkit API Reference for more details.

The tutorial notebooks provide examples that demonstrate the various workflows.

  1. Download the resource using the NGC CLI.


    ngc registry resource download-version "nvidia/tao/tao-getting-started:5.2.0"

  2. Find tutorial notebooks (.ipynb files).


    cd tao-getting-started_v5.2.0/notebooks/tao_api_starter_kit/api

  3. Serve these Notebook examples using Jupyter-lab pods.


    Jupyter-lab pods are not multi-tenant and present a security risk, as the user could gain access to manage the whole GPU cluster within the pod.

    Also, the instructions mentioned below to forward the port and use the Jupyter-lab pod is only meant to show how to launch the TAO API notebooks across different Cloud Service Providers (CSPs) in a cloud agnostic way. Each organization will have it’s own specific security policies regarding opening cloud service ports publicly, so make sure you review them and comply before executing the below mentioned steps.

    TAO Toolkit version 5.2 provides a Jupyter-lab pod with the dependencies required to launch notebooks pre-installed.

    1. On the remote machine where the one-click deployment scripts are run, execute the following command. Note that this command will not terminate automatically, therefore to run any new commands on this machine, you will need to open a new terminal.


      kubectl port-forward service/tao-toolkit-api-jupyterlab-service :8888

      Example output


      Forwarding from -> 8888 Forwarding from [::1]:33465 -> 8888

    2. Copy the port number following the address from output of previous step.

    3. In a new terminal window on your local machine, launch ssh tunneling of the remote machine via the following command. user_name and IP_address corresponds to the machine where the deployment scripts are run. Note that this command will not terminate automatically.


      ssh -N -L <port_number_copied>:localhost:<port_number_copied> <user_name>@<IP_address>

    4. On the browser of the local machine, visit the address localhost:<port_number_copied>/notebook/lab

      Now, the jupyterlab session inside the jupyter-lab pod is accessible to you on the browser.
      You can run api or TAO-Client notebooks inside the tao_end2end folder.

    5. Inside the notebook, for FIXME of ip_address, use ingress-nginx-controller and for FIXME of port_number, use 80.

    You can view the pod name using kubectl get pods -n default | grep "jupyter".

    You can copy files into the pod using kubectl cp <path to local file system> <jupyterlab-pod pod name>:<path inside jupyterlab-pod>

    You can also launch the notebooks without the Jupyter-lab pod by installing the Jupyter notebook pip package and using that package to launch notebooks from the getting started directory: Instead of using NGC, The tutorial notebooks can also be downloaded from the machine where TAO API is deployed. You can obtain them using:


    wget https://<ip_of_hosted_machine>:<nginx_service_forwarded_port>/ unzip cd api pip3 install jupyter notebook jupyter notebook --ip

Previous Deployment
Next Remote Client
© Copyright 2024, NVIDIA. Last updated on Mar 18, 2024.