TAO Toolkit Quick Start Guide

This page provides a video and text-based quick start guide for installing and running TAO Toolkit.

Hardware

The following system configuration is recommended to achieve reasonable training performance with TAO Toolkit and supported models provided:

• 32 GB system RAM

• 32 GB of GPU RAM

• 8 core CPU

• 1 NVIDIA GPU

• 100 GB of SSD space

TAO Toolkit is supported on discrete GPUs, such as A100, A40, A30, A2, A16, A100x, A30x, V100, T4, Titan-RTX and Quadro-RTX.

Software Requirements

Installing the Pre-requisites

The tao-launcher is strictly a python3 only package, capable of running on python 3.6.9 or 3.7.

1. Install docker-ce by following the official instructions.

   Once you have installed docker-ce, follow the post-installation steps to ensure that the docker can be run without sudo.

2. Install nvidia-container-toolkit by following the install-guide.

3. Get an NGC account and API key:

   1. Go to NGC and click the TAO Toolkit container in the Catalog tab. This message is displayed: “Sign in to access the PULL feature of this repository”.

   2. Enter your Email address and click Next, or click Create an Account.

   3. Choose your organization when prompted for Organization/Team.

   4. Click Sign In.

4. Log in to the NGC docker registry (nvcr.io) using the command docker login nvcr.io and enter the following credentials:

   Copy

   Copied!

               
   
               
   a. Username: "$oauthtoken"
   b. Password: "YOUR_NGC_API_KEY"
           

   
   

   where YOUR_NGC_API_KEY corresponds to the key you generated from step 3.

Installing TAO Toolkit

TAO Toolkit is a Python pip package that is hosted on the NVIDIA PyIndex. The package uses the docker restAPI under the hood to interact with the NGC Docker registry to pull and instantiate the underlying docker containers. You must have an NGC account and an API key associated with your account. See the Installation Prerequisites section for details on creating an NGC account and obtaining an API key.

1. Create a new conda environment using miniconda.

   1. Follow the instructions in this link to set up a conda environemnt using a miniconda.

   2. Once you have installed miniconda, create a new environment by setting the Python version to 3.6.

      Copy

      Copied!

                  
      
                  
      conda create -n launcher python=3.6
              

   3. Activate the conda environment that you have just created.

      Copy

      Copied!

                  
      
                  
      conda activate launcher
              

   4. Once you have activated your conda environement, the command prompt should show the name of your conda environment.

      Copy

      Copied!

                  
      
                  
      (launcher) py-3.6.9 desktop:
              

   5. When you are done with you session, you may deactivate your conda environemnt using the deactivate command:

      Copy

      Copied!

                  
      
                  
      conda deactivate
              

   6. You may re-instantiate this created conda environemnt using the following command.

   Copy

   Copied!

               
   
               
   conda activate launcher
           

   
   

2. Install the TAO Launcher Python package called nvidia-tao.

   Copy

   Copied!

               
   
               
   pip3 install nvidia-tao
           

   Note

   If you had installed an older version of nvidia-tao launcher, you may upgrade to the latest version by running the following command.

   Copy

   Copied!

               
   
               
   pip3 install --upgrade nvidia-tao
           

3. Invoke the entrypoints using the tao command.

   Copy

   Copied!

               
   
               
   tao --help
           

   The sample output of the above command is:

   Copy

   Copied!

               
   
               
   usage: tao [-h]
            {list,stop,info,augment,bpnet,classification,detectnet_v2,dssd,emotionnet,faster_rcnn,fpenet,gazenet,gesturenet,
            heartratenet,intent_slot_classification,lprnet,mask_rcnn,punctuation_and_capitalization,question_answering,
            retinanet,speech_to_text,ssd,text_classification,converter,token_classification,unet,yolo_v3,yolo_v4,yolo_v4_tiny}
            ...
   
   Launcher for TAO
   
   optional arguments:
   -h, --help            show this help message and exit
   
   tasks:
         {list,stop,info,augment,bpnet,classification,detectnet_v2,dssd,emotionnet,faster_rcnn,fpenet,gazenet,gesturenet,heartratenet
         ,intent_slot_classification,lprnet,mask_rcnn,punctuation_and_capitalization,question_answering,retinanet,speech_to_text,
         ssd,text_classification,converter,token_classification,unet,yolo_v3,yolo_v4,yolo_v4_tiny}
           

   Note that under tasks you can see all the launcher-invokable tasks. The following are the specific tasks that help with handling the launched commands using the TAO Launcher:

   • list

   • stop

   • info

            

            
export PATH=$PATH:~/.local/bin
        

Downloading Jupyter Noteboks and Resources

Downloading the sample notebooks

Example Jupyter notebooks for all the tasks that are supported in TAO Toolkit are available in NGC resources. TAO Toolkit provides sample workflows for Computer Vision and Conversational AI.

Computer Vision

All the samples for the supported computer vision tasks are hosted on ngc under the TAO Computer Vision Samples. To run the available examples, download this sample resource by using the following commands.

            

            
wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.4.0/zip -O cv_samples_v1.4.0.zip
unzip -u cv_samples_v1.4.0.zip  -d ./cv_samples_v1.4.0 && rm -rf cv_samples_v1.4.0.zip && cd ./cv_samples_v1.4.0
        

Purpose-Built Pre-trained Models

The following is a list of purpose-built pretrained models mapped with their corresponding samples.

Open model architecture

Conversational AI

The TAO Conversational AI package, provides several end to end sample workflows to train conversational AI models using TAO Toolkit and subsequently deploying them to Riva. You can find these samples at:

You can download these resources, by using the NGC CLI command available at the NGC resource page. Once you download the respective tutorial resource, you may instantiate the jupyter notebook server.

            

            
pip3 install jupyter
jupyter notebook --ip 0.0.0.0 --allow-root --port 8888
        

Copy and paste the link produced from this command into your browser to access the notebook. The /workspace/examples folder will contain a demo notebook. Feel free to use any free port available to host the notebook if port 8888 is unavailable.

Downloading the Models

The TAO Toolkit Docker gives you access to a repository of pretrained models that can serve as a starting point when training deep neural networks. These models are hosted on the NGC. To download the models, please download the NGC CLI and install it. More information about the NGC Catalog CLI is available here. Once you have installed the CLI, you may follow the instructions below to configure the NGC CLI and download the models.

Listing all available models

Use this command to get a list of models that are hosted in the NGC model registry:

            

            
ngc registry model list <model_glob_string>
        

Here is an example of using this command for the computer vision models:

            

            
ngc registry model list nvidia/tao/pretrained_*
        

This command gives us a list of the pretrained backbones available for different tasks:

• Classification

• Object Detection with Detectnet_v2

• Object Detection with SSD/DSSD/YOLOv3/YOLOv4/YOLOv4-Tiny/FasterRCNN/RetinaNet

• Object Detection with EfficientDet

• Instance Segmentation

• Semantic Segmentation

To view the full list of computer vision and conversational AI models, use the following command:

            

            
ngc registry model list nvidia/tao/*
        

Downloading a model

Use this command to download the model you have chosen from the NGC model registry:

            

            
ngc registry model download-version <org/team/model_name:version> -dest <path_to_download_dir>
        

For example, use this command to download the resnet 18 classification model to the $USER_EXPERIMENT_DIR directory:

            

            
ngc registry model download-version
nvidia/tao/pretrained_classification:resnet18 --dest $USER_EXPERIMENT_DIR/pretrained_resnet18
        

            

            
Downloaded 82.41 MB in 9s, Download speed: 9.14 MB/s
----------------------------------------------------
Transfer id: pretrained_classification_vresnet18 Download status: Completed.
Downloaded local path: /workspace/tao-experiments/pretrained_resnet18/
Total files downloaded: 2
Total downloaded size: 82.41 MB
Started at: 2019-07-16 01:29:53.028400
Completed at: 2019-07-16 01:30:02.053016
Duration taken: 9s seconds
        

Once you have downloaded the notebook samples and required pretrained models, you can start the respective sample notebook with the following command:

            

            
jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root
        

Open an internet browser on localhost and navigate to the following URL:

            

            
http://0.0.0.0:8888
        

Execute the cells in the notebook to train a model using TAO Toolkit.

Once training is complete, follow these instructions to deploy a computer vision model to DeepStream. For Conversational AI models, follow the instructions in this section.