TAO Toolkit Quick Start Guide

This page provides a quick start guide for installing and running TAO Toolkit.

Requirements

Hardware

The following system configuration is recommended to achieve reasonable training performance with TAO Toolkit and supported models provided:

  • 32 GB system RAM

  • 32 GB of GPU RAM

  • 8 core CPU

  • 1 NVIDIA GPU

  • 100 GB of SSD space

TAO Toolkit is supported on A100, V100 and RTX 30x0 GPUs.

Software Requirements

Software

Version

Ubuntu 18.04 LTS

18.04

python

>=3.6.9

docker-ce

>19.03.5

docker-API

1.40

nvidia-container-toolkit

>1.3.0-1

nvidia-container-runtime

3.4.0-1

nvidia-docker2

2.5.0-1

nvidia-driver

>455

python-pip

>21.06

python-dev

nvidia-pyindex

Installing the Pre-requisites

The tao-launcher is strictly a python3 only package, capable of running on python 3.6.9 or 3.7.

  1. Install docker-ce by following the official instructions.

    Once you have installed docker-ce, follow the post-installation steps to ensure that the docker can be run without sudo.

  2. Install nvidia-container-toolkit by following the install-guide.

  3. Get an NGC account and API key:

    1. Go to NGC and click the TAO Toolkit container in the Catalog tab. This message is displayed: “Sign in to access the PULL feature of this repository”.

    2. Enter your Email address and click Next, or click Create an Account.

    3. Choose your organization when prompted for Organization/Team.

    4. Click Sign In.

  4. Log in to the NGC docker registry (nvcr.io) using the command docker login nvcr.io and enter the following credentials:

    a. Username: "$oauthtoken"
    b. Password: "YOUR_NGC_API_KEY"
    

    where YOUR_NGC_API_KEY corresponds to the key you generated from step 3.

Note

DeepStream 6.0 - NVIDIA SDK for IVA inference is recommended.

Installing TAO Toolkit

TAO Toolkit is a Python pip package that is hosted on the NVIDIA PyIndex. The package uses the docker restAPI under the hood to interact with the NGC Docker registry to pull and instantiate the underlying docker containers. You must have an NGC account and an API key associated with your account. See the Installation Prerequisites section for details on creating an NGC account and obtaining an API key.

  1. Create a new virtualenv using virtualenvwrapper.

    You may follow the instructions in this link to set up a Python virtualenv using a virtualenvwrapper.

    Once you have followed the instructions to install virtualenv and virtualenvwrapper, set the Python version in the virtualenv. This can be done in either of the following ways:

    • Defining the environment variable called VIRTUALENVWRAPPER_PYTHON. This variable should point to the path where the python3 binary is installed in your local machine. You can also add it to your .bashrc or .bash_profile for setting your Python virtualenv by default.

      export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
      
    • Setting the path to the python3 binary when creating your virtualenv using the virtualenvwrapper wrapper

      mkvirtualenv launcher -p /path/to/your/python3
      

    Once you have logged into the virtualenv, the command prompt should show the name of your virtual environment

    (launcher) py-3.6.9 desktop:
    

    When you are done with you session, you may deactivate your virtualenv using the deactivate command:

    deactivate
    

    You may re-instantiate this created virtualenv env using the workon command.

    workon launcher
    
  2. Install the TAO Launcher Python package called nvidia-tao.

    pip3 install nvidia-pyindex
    pip3 install nvidia-tao
    

    Note

    The nvidia-tao package is hosted in the nvidia-pyindex, which has to be installed as a pre-requisite to install nvidia-tao.

    If you had installed an older version of nvidia-tao launcher, you may upgrade to the latest version by running the following command.

    pip3 install --upgrade nvidia-tao
    
  3. Invoke the entrypoints using the tao command.

    tao --help
    

    The sample output of the above command is:

    usage: tao [-h]
             {list,stop,info,augment,bpnet,classification,detectnet_v2,dssd,emotionnet,faster_rcnn,fpenet,gazenet,gesturenet,
             heartratenet,intent_slot_classification,lprnet,mask_rcnn,punctuation_and_capitalization,question_answering,
             retinanet,speech_to_text,ssd,text_classification,converter,token_classification,unet,yolo_v3,yolo_v4,yolo_v4_tiny}
             ...
    
    Launcher for TAO
    
    optional arguments:
    -h, --help            show this help message and exit
    
    tasks:
          {list,stop,info,augment,bpnet,classification,detectnet_v2,dssd,emotionnet,faster_rcnn,fpenet,gazenet,gesturenet,heartratenet
          ,intent_slot_classification,lprnet,mask_rcnn,punctuation_and_capitalization,question_answering,retinanet,speech_to_text,
          ssd,text_classification,converter,token_classification,unet,yolo_v3,yolo_v4,yolo_v4_tiny}
    

    Note that under tasks you can see all the launcher-invokable tasks. The following are the specific tasks that help with handling the launched commands using the TAO Launcher:

    • list

    • stop

    • info

Note

When installing the TAO Toolkit Launcher to your host machine’s native python3 as opposed to the recommended route of using virtual environment, you may get an error saying that tao binary wasn’t found. This is because the path to your tao binary installed by pip wasn’t added to the PATH environment variable in your local machine. In this case, please run the following command:

export PATH=$PATH:~/.local/bin

Running the TAO Toolkit

Information about the TAO Launcher CLI and details on using it to run TAO supported tasks are captured in the section.

Use the examples

Example Jupyter notebooks for all the tasks that are supported in TAO Toolkit are available in NGC resources. TAO Toolkit provides sample workflows for Computer Vision and Conversational AI.

Computer Vision

All the samples for the supported computer vision tasks are hosted on ngc under the TAO Computer Vision Samples. To run the available examples, download this sample resource by using the following commands.

wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.3.0/zip -O cv_samples_v1.3.0.zip
unzip -u cv_samples_v1.3.0.zip  -d ./cv_samples_v1.3.0 && rm -rf cv_samples_v1.3.0.zip && cd ./cv_samples_v1.3.0

Conversational AI

The TAO Conversational AI package, provides several end to end sample workflows to train conversational AI models using TAO Toolkit and subsequently deploying them to Riva. You can find these samples at:

Conversational AI Task

Jupyter Notebooks

Speech to Text

Speech to Text Notebook

Speech to Text Citrinet

Speech to Text Citrinet Notebook

Question Answering

Question Answering Notebook

Text Classification

Text Classification Notebook

Token Classification

Token Classification Notebook

Punctuation and Capitalization

Punctuation Capitalization Notebook

Intent and Slot Classification

Intent Slot Classification Notebook

NGram Language Model

NGram Language Model Notebook

Text to Speech

Text to Speech Notebook

You can download these resources, by using the NGC CLI command available at the NGC resource page. Once you download the respective tutorial resource, you may instantiate the jupyter notebook server.

pip3 install jupyter
jupyter notebook --ip 0.0.0.0 --allow-root --port 8888

Copy and paste the link produced from this command into your browser to access the notebook. The /workspace/examples folder will contain a demo notebook. Feel free to use any free port available to host the notebook if port 8888 is unavailable.

Downloading the Models

The TAO Toolkit Docker gives you access to a repository of pretrained models that can serve as a starting point when training deep neural networks. These models are hosted on the NGC. To download the models, please download the NGC CLI and install it. More information about the NGC Catalog CLI is available here. Once you have installed the CLI, you may follow the instructions below to configure the NGC CLI and download the models.

Configure the NGC API key

Using the NGC API Key obtained in Installation Prerequisites, configure the enclosed ngc cli by executing this command and following the prompts:

ngc config set

Get a list of models

Use this command to get a list of models that are hosted in the NGC model registry:

ngc registry model list <model_glob_string>

For the computer vision models, here is an example of using this command:

ngc registry model list nvidia/tao/pretrained_*

Note

All our classification models have names based on this template: nvidia/tao/pretrained_classification:<template>.

To view all the conversational AI models, you may using the following command:

ngc registry model list nvidia/tlt-riva/*

Download a model

Use this command to download the model you have chosen from the NGC model registry:

ngc registry model download-version <ORG/model_name:version> -dest <path_to_download_dir>

For example, use this command to download the resnet 18 classification model to the $USER_EXPERIMENT_DIR directory:

ngc registry model download-version
nvidia/tao/pretrained_classification:resnet18 --dest
$USER_EXPERIMENT_DIR/pretrained_resnet18
Downloaded 82.41 MB in 9s, Download speed: 9.14 MB/s
----------------------------------------------------
Transfer id: tlt_iva_classification_resnet18_v1 Download status: Completed.
Downloaded local path: /workspace/tao-experiments/pretrained_resnet18/
tlt_resnet18_classification_v1
Total files downloaded: 2
Total downloaded size: 82.41 MB
Started at: 2019-07-16 01:29:53.028400
Completed at: 2019-07-16 01:30:02.053016
Duration taken: 9s seconds

Training with Jupyter Notebook

Install the Python Virtual Environment

Follow these instructions to install the virtualenv with Python 3.6.9. Once virtualenvwrapper is set up, set the version of python to be used in the virtual env by using the VIRTUALENVWRAPPER_PYTHON variable. You may do so by running the following:

export VIRTUALENVWRAPPER_PYTHON=/path/to/bin/python3.x

where x >= 6 and <= 8.

Use the following command to instantiate a virtual environment:

mkvirtualenv launcher -p /path/to/bin/python3.x

where x >= 6 and <= 8

Download Jupyter Notebook

TAO Toolkit provides samples notebooks to walk through and prescrible TAO workflow. These samples are hosted on NGC as a resource and can be downloaded from NGC by executing the command mentioned below.

wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.3.0/zip -O cv_samples_v1.3.0.zip
unzip -u cv_samples_v1.3.0.zip  -d ./cv_samples_v1.3.0 && rm -rf cv_samples_v1.3.0.zip && cd ./cv_samples_v1.3.0

The list with their corresponding samples mapped are mentioned below.

Model Name

Jupyter Notebook

VehicleTypeNet

classification/classification.ipynb

VehicleMakeNet

classification/classification.ipynb

TrafficCamNet

detectnet_v2/detectnet_v2.ipynb

PeopleSegNet

mask_rcnn/mask_rcnn.ipynb

PeopleNet

detectnet_v2/detectnet_v2.ipynb

License Plate Recognition

lprnet/lprnet.ipynb

License Plate Detection

detectnet_v2/detectnet_v2.ipynb

Heart Rate Estimation

heartratenet/heartratenet.ipynb

Gesture Recognition

gesturenet/gesturenet.ipynb

Gaze Estimation

gazenet/gazenet.ipynb

Facial Landmark

fpenet/fpenet.ipynb

FaceDetectIR

detectnet_v2/detectnet_v2.ipynb

FaceDetect

facenet/facenet.ipynb

Emotion Recognition

emotionnet/emotionnet.ipynb

DashCamNet

detectnet_v2/detectnet_v2.ipynb

BodyPoseNet

bpnet/bpnet.ipynb

ActionRecognitionNet

actionrecognitionnet/actionrecognitionnet.ipynb

Open model architecture:

Jupyter notebook

DetectNet_v2

detectnet_v2/detectnet_v2.ipynb

FasterRCNN

faster_rcnn/faster_rcnn.ipynb

YOLOv3

yolo_v3/yolo_v3.ipynb

YOLOv4

yolo_v4/yolo_v4.ipynb

YOLOv4-tiny

yolo_v4_tiny/yolo_v4_tiny.ipynb

SSD

ssd/ssd.ipynb

DSSD

dssd/dssd.ipynb

RetinaNet

retinanet/retinanet.ipynb

MaskRCNN

mask_rcnn/mask_rcnn.ipynb

UNET

unet/unet_isbi.ipynb

Classification

classification/classification.ipynb

EfficientDet

efficientdet/efficientdet.ipynb

Start Jupyter Notebook

Once the notebook samples are downloaded, you may start the notebook using the below commands:

jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root

Open an internet browser on localhost and navigate to the following URL:

http://0.0.0.0:8888

Note

If you want to run the notebook from a remote server then please follow these steps.

1. Train the Model

Follow the Notebook instructions to train the model.

You may now deploy these models to DeepStream by following the instructions here.