TAO Toolkit Quick Start Guide
This page provides a video and text-based quick start guide for installing and running TAO Toolkit.
Hardware
The following system configuration is recommended to achieve reasonable training performance with TAO Toolkit and supported models provided:
32 GB system RAM
32 GB of GPU RAM
8 core CPU
1 NVIDIA GPU
100 GB of SSD space
TAO Toolkit is supported on discrete GPUs, such as A100, A40, A30, A2, A16, A100x, A30x, V100, T4, Titan-RTX and Quadro-RTX.
TAO Toolkit is not supported on GPU’s before the Pascal generation.
Software Requirements
Software |
Version |
** Comment** |
Ubuntu LTS |
20.04 |
|
python |
>=3.6.9<3.7 |
Not needed if you use TAO toolkit API |
docker-ce |
>19.03.5 |
Not needed if you use TAO toolkit API |
docker-API |
1.40 |
Not needed if you use TAO toolkit API |
|
>1.3.0-1 |
Not needed if you use TAO toolkit API |
nvidia-container-runtime |
3.4.0-1 |
Not needed if you use TAO toolkit API |
nvidia-docker2 |
2.5.0-1 |
Not needed if you use TAO toolkit API |
nvidia-driver |
>520 |
Not needed if you use TAO toolkit API |
python-pip |
>21.06 |
Not needed if you use TAO toolkit API |
Package Content
Download the TAO package which contains startup scripts, Jupyter notebooks and config files. TAO is supported on Google Colab; if you want to try on Colab, you can skip this step and directly scroll down to the running in colab section.
wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/tao-getting-started/versions/4.0.0/zip -O getting_started_v4.0.0.zip
unzip -u getting_started_v4.0.0.zip -d ./getting_started_v4.0.0 && rm -rf getting_started_v4.0.0.zip && cd ./getting_started_v4.0.0
TAO toolkit is available as a docker container or a collection of python wheels. There are 4 ways to run TAO Toolkit depending on your preference and setup, through
Launcher CLI
The TAO Toolkit launcher is a lightweight Python based command-line interface. The launcher acts as a front-end for TAO Toolkit containers built on top of PyTorch and TensorFlow. The CLI abstracts the user from the information about which network is implemented in what container. The respective container gets launcher automatically based on the model you plan to use.
To get started with the launcher, follow the instructions below to install the required pre-requisite software.
Installing the Pre-requisites
The TAO Toolkit launcher is strictly a python3 only package, capable of running on python versions >= 3.6.9.
Install
docker-ce
by following the official instructions.Once you have installed docker-ce, follow the post-installation steps to ensure that the docker can be run without
sudo
.Install
nvidia-container-toolkit
by following the install-guide.Get an NGC account and API key:
Go to NGC and click the TAO Toolkit container in the Catalog tab. This message is displayed: “Sign in to access the PULL feature of this repository”.
Enter your Email address and click Next, or click Create an Account.
Choose your organization when prompted for Organization/Team.
Click Sign In.
Log in to the NGC docker registry (
nvcr.io
) using the commanddocker login nvcr.io
and enter the following credentials:a. Username: "$oauthtoken" b. Password: "YOUR_NGC_API_KEY"
where
YOUR_NGC_API_KEY
corresponds to the key you generated from step 3.
DeepStream 6.0 - NVIDIA SDK for IVA inference is recommended.
Setup your python environment with python version >= 3.6.9
NoteNVIDIA recommends setting up a python environment using
miniconda
. The following instructions show how to setup a pythonconda
environment.Follow the instructions in this link to set up a conda environment using a miniconda.
Once you have installed
miniconda
, create a new environment by setting the Python version to 3.6.conda create -n launcher python=3.6
Activate the
conda
environment that you have just created.conda activate launcher
Once you have activated your
conda
environment, the command prompt should show the name of your conda environment.(launcher) py-3.6.9 desktop:
When you are done with you session, you may deactivate your
conda
environment using thedeactivate
command:conda deactivate
You may re-instantiate this created
conda
environment using the following command.
conda activate launcher
Installing TAO Launcher
Once you have installed the required pre-requisites.
Install the CLI launcher via the quick start script downloaded with the getting_started NGC package from here
bash setup/quickstart_launcher.sh --install
You can also use this script to update the launcher to the latest version of TAO Toolkit by running the following command
bash setup/quickstart_launcher.sh --upgrade
Invoke the entrypoints using the
tao
command.tao --help
The sample output of the above command is:
usage: tao [-h] {list,stop,info,augment,bpnet,classification,detectnet_v2,dssd,emotionnet,faster_rcnn,fpenet,gazenet,gesturenet, heartratenet,intent_slot_classification,lprnet,mask_rcnn,punctuation_and_capitalization,question_answering, retinanet,speech_to_text,ssd,text_classification,converter,token_classification,unet,yolo_v3,yolo_v4,yolo_v4_tiny} ... Launcher for TAO optional arguments: -h, --help show this help message and exit tasks: {list,stop,info,augment,bpnet,classification,detectnet_v2,dssd,emotionnet,faster_rcnn,fpenet,gazenet,gesturenet,heartratenet ,intent_slot_classification,lprnet,mask_rcnn,punctuation_and_capitalization,question_answering,retinanet,speech_to_text, ssd,text_classification,converter,token_classification,unet,yolo_v3,yolo_v4,yolo_v4_tiny}
Note that under tasks you can see all the launcher-invokable tasks. The following are the specific tasks that help with handling the launched commands using the TAO Launcher:
list
stop
info
When installing the TAO Toolkit Launcher to your host machine’s native python3 as opposed to the recommended route of using virtual
environment, you may get an error saying that tao
binary wasn’t found. This is because the path to your tao
binary
installed by pip wasn’t added to the PATH
environment variable in your local machine. In this case, please run the
following command:
export PATH=$PATH:~/.local/bin
Running from container
Users have option to also run TAO directly using the docker container. To use container directly, user needs to know which container to pull. There are multiple containers under TAO, and depending on the model that you want to train you will need to pull the appropriate container. This is not required when using the Launcher CLI.
export DOCKER_REGISTRY="nvcr.io"
export DOCKER_NAME="nvidia/tao/tao-toolkit"
export DOCKER_TAG="4.0.0-tf1.15.5" ## for TensorFlow docker
export DOCKER_CONTAINER=$DOCKER_REGISTRY/$DOCKER_NAME:$DOCKER_TAG
docker run -it --rm --gpus all -v /path/in/host:/path/in/docker $DOCKER_CONTAINER \
detectnet_v2 train -e /path/to/experiment/spec.txt -r /path/to/results/dir -k $KEY --gpus 4
For detailed instructions on how to run directly from containers, refer to this section
Running TAO Toolkit APIs
TAO Toolkit API is a Kubernetes service that enables building end-to-end AI models using REST APIs. The API service can be installed on a Kubernetes cluster (local / AWS EKS) using a Helm chart along with minimal dependencies. TAO toolkit jobs can be run using GPUs available on the cluster and can scale to a multi-node setting. Users can use a TAO client CLI to interact with TAO services remotely or can integrate it in their own apps and services directly using REST APIs.
To get started, use the provided one-click deploy script to deploy either on bare-metal setup or on
managed Kubernetes service like Amazon EKS. Jupyter notebooks to train using the APIs directly or using
the client app is provided under notebooks/api_starter_kit
.
bash setup/quickstart_api_bare_metal
bash setup/quickstart_api_aws_eks
More information about setting up the API services and the API is provided in here.
Running from python wheels
You can also run TAO directly on bare-metal without docker or K8s by using the python wheels. As a sample tutorial, TAO Toolkit provides sample tutorials to deploy directly on Google Colab without having to configure your infrastructure. The full instructions on how to work with google colab are provided here.
As a part of the TAO toolkit package, example jupyter notebooks are available for all the tasks that supported in TAO.
Computer vision
All computer vision samples are included in the getting started resource on NGC.
Purpose-Built Pre-trained Models
The following is a list of purpose-built pretrained models mapped with their corresponding samples.
Model Name |
Jupyter Notebook |
---|---|
VehicleTypeNet |
notebooks/tao_launcher_starter_kit/classification/classification.ipynb |
VehicleMakeNet |
notebooks/tao_launcher_starter_kit/classification/classification.ipynb |
TrafficCamNet |
notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb |
PeopleSegNet |
notebooks/tao_launcher_starter_kit/mask_rcnn/mask_rcnn.ipynb |
PeopleNet |
notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb |
License Plate Recognition |
notebooks/tao_launcher_starter_kit/lprnet/lprnet.ipynb |
License Plate Detection |
notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb |
Heart Rate Estimation |
notebooks/tao_launcher_starter_kit/heartratenet/heartratenet.ipynb |
Gesture Recognition |
notebooks/tao_launcher_starter_kit/gesturenet/gesturenet.ipynb |
Gaze Estimation |
notebooks/tao_launcher_starter_kit/gazenet/gazenet.ipynb |
Facial Landmark |
notebooks/tao_launcher_starter_kit/fpenet/fpenet.ipynb |
FaceDetectIR |
notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb |
FaceDetect |
notebooks/tao_launcher_starter_kit/facenet/facenet.ipynb |
Emotion Recognition |
notebooks/tao_launcher_starter_kit/emotionnet/emotionnet.ipynb |
DashCamNet |
notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb |
BodyPoseNet |
notebooks/tao_launcher_starter_kit/bpnet/bpnet.ipynb |
ActionRecognitionNet |
notebooks/tao_launcher_starter_kit/actionrecognitionnet/actionrecognitionnet.ipynb |
PoseClassificationNet |
notebooks/tao_launcher_starter_kit/pose_classification_net/poseclassificationnet.ipynb |
PointPillars |
notebooks/tao_launcher_starter_kit/pointpillars/pointpillars.ipynb |
ReIdentification |
notebooks/tao_launcher_starter_kit/re_identification_net/reidentificationnet.ipynb |
Open model architecture
Network Architecture |
Jupyter Notebook |
---|---|
DetectNet_v2 |
notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb |
FasterRCNN |
notebooks/tao_launcher_starter_kit/faster_rcnn/faster_rcnn.ipynb |
YOLOv3 |
notebooks/tao_launcher_starter_kit/yolo_v3/yolo_v3.ipynb |
YOLOv4 |
notebooks/tao_launcher_starter_kit/yolo_v4/yolo_v4.ipynb |
YOLOv4-tiny |
notebooks/tao_launcher_starter_kit/yolo_v4_tiny/yolo_v4_tiny.ipynb |
SSD |
notebooks/tao_launcher_starter_kit/ssd/ssd.ipynb |
DSSD |
notebooks/tao_launcher_starter_kit/dssd/dssd.ipynb |
RetinaNet |
notebooks/tao_launcher_starter_kit/retinanet/retinanet.ipynb |
MaskRCNN |
notebooks/tao_launcher_starter_kit/mask_rcnn/mask_rcnn.ipynb |
UNET |
notebooks/tao_launcher_starter_kit/unet/unet_isbi.ipynb |
Classification (TF1) |
notebooks/tao_launcher_starter_kit/classification_tf1/classification.ipynb |
Classification (TF2) |
notebooks/tao_launcher_starter_kit/classification_tf2/classification.ipynb |
EfficientDet (TF1) |
notebooks/tao_launcher_starter_kit/efficientdet_tf1/efficientdet.ipynb |
EfficientDet (TF2) |
notebooks/tao_launcher_starter_kit/efficientdet_tf2/efficientdet.ipynb |
PointPillars |
notebooks/tao_launcher_starter_kit/pointpillars/pointpillars.ipynb |
Conversational AI
The TAO Conversational AI package, provides several end to end sample workflows to train conversational AI models using TAO Toolkit and subsequently deploying them to Riva. You can find these samples at:
Conversational AI Task |
Jupyter Notebooks |
---|---|
Speech to Text Citrinet |
|
Speech to Text Conformer |
|
Question Answering |
|
Text Classification |
|
Token Classification |
|
Punctuation and Capitalization |
|
Intent and Slot Classification |
|
NGram Language Model |
|
Text to Speech |
You can download these resources, by using the NGC CLI command available at the NGC resource page. Once you download the respective tutorial resource, you may instantiate the jupyter notebook server.
pip3 install jupyter
jupyter notebook --ip 0.0.0.0 --allow-root --port 8888
Copy and paste the link produced from this command into your browser to access the notebook. The /workspace/examples folder will contain a demo notebook. Feel free to use any free port available to host the notebook if port 8888 is unavailable.
The TAO Toolkit Docker gives you access to a repository of pretrained models that can serve as a starting point when training deep neural networks. These models are hosted on the NGC. To download the models, please download the NGC CLI and install it. More information about the NGC Catalog CLI is available here. Once you have installed the CLI, you may follow the instructions below to configure the NGC CLI and download the models.
Listing all available models
Use this command to get a list of models that are hosted in the NGC model registry:
ngc registry model list <model_glob_string>
Here is an example of using this command for the computer vision models:
ngc registry model list nvidia/tao/pretrained_*
This command gives us a list of the pretrained backbones available for different tasks:
Classification
Object Detection with Detectnet_v2
Object Detection with SSD/DSSD/YOLOv3/YOLOv4/YOLOv4-Tiny/FasterRCNN/RetinaNet
Object Detection with EfficientDet
Instance Segmentation
Semantic Segmentation
All our classification models have names based on this template:
nvidia/tao/pretrained_classification:<template>
.
To view the full list of computer vision and conversational AI models, use the following command:
ngc registry model list nvidia/tao/*
Downloading a model
Use this command to download the model you have chosen from the NGC model registry:
ngc registry model download-version <org/team/model_name:version> -dest <path_to_download_dir>
For example, use this command to download the resnet 18 classification model to the
$USER_EXPERIMENT_DIR
directory:
ngc registry model download-version
nvidia/tao/pretrained_classification:resnet18 --dest $USER_EXPERIMENT_DIR/pretrained_resnet18
Downloaded 82.41 MB in 9s, Download speed: 9.14 MB/s
----------------------------------------------------
Transfer id: pretrained_classification_vresnet18 Download status: Completed.
Downloaded local path: /workspace/tao-experiments/pretrained_resnet18/
Total files downloaded: 2
Total downloaded size: 82.41 MB
Started at: 2019-07-16 01:29:53.028400
Completed at: 2019-07-16 01:30:02.053016
Duration taken: 9s seconds
Once you have downloaded the notebook samples and required pretrained models, you can start the respective sample notebook with the following command:
jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root
Open an internet browser on localhost and navigate to the following URL:
http://0.0.0.0:8888
If you want to run the notebook from a remote server then please follow these steps.
Execute the cells in the notebook to train a model using TAO Toolkit.
Once training is complete, follow these instructions to deploy a computer vision model to DeepStream. For Conversational AI models, follow the instructions in this section.