TAO Toolkit v5.3.0
NVIDIA TAO v5.3.0

TAO Toolkit Quick Start Guide

This page provides a video and text-based quick start guide for installing and running TAO Toolkit.

Hardware Requirements

Minimum System Configuration

  • 8 GB system RAM

  • 4 GB of GPU RAM

  • 8 core CPU

  • 1 NVIDIA GPU

  • 100 GB of SSD space

Recommended System Configuration

  • 32 GB system RAM

  • 32 GB of GPU RAM

  • 8 core CPU

  • 1 NVIDIA GPU

  • 100 GB of SSD space

TAO Toolkit is supported on discrete GPUs, such as H100, A100, A40, A30, A2, A16, A100x, A30x, V100, T4, Titan-RTX, and Quadro-RTX.

Note

TAO Toolkit is not supported on GPU’s before the Pascal generation.

Software Requirements

Software Version ** Comment**
Ubuntu LTS 20.04
python >3.7,<=3.10 Not needed if you use TAO toolkit API
docker-ce >19.03.5 Not needed if you use TAO toolkit API
docker-API 1.40 Not needed if you use TAO toolkit API
nvidia-container-toolkit >1.3.0-1 Not needed if you use TAO toolkit API
nvidia-container-runtime 3.4.0-1 Not needed if you use TAO toolkit API
nvidia-docker2 2.5.0-1 Not needed if you use TAO toolkit API
nvidia-driver >535.xx Not needed if you use TAO toolkit API
python-pip >21.06 Not needed if you use TAO toolkit API

Getting Started

Download the TAO package that contains startup scripts, Jupyter Notebooks, and config files. TAO is supported on Google Colab; if you want to try on Colab, you can skip this step and scroll down to the running in colab section.

Copy
Copied!
            

wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/tao-getting-started/versions/5.3.0/zip -O getting_started_v5.3.0.zip unzip -u getting_started_v5.3.0.zip -d ./getting_started_v5.3.0 && rm -rf getting_started_v5.3.0.zip && cd ./getting_started_v5.3.0

File Hierarchy and Overview

The TAO Toolkit getting started resource is broadly classified into two components.

  • setup: These are a set of quick start scripts to help install and deploy the TAO Toolkit launcher and the TAO Toolkit API’s in various Cloud Service Providers.

  • notebooks: These are beginner friendly end-to-end tutorial notebooks, that help you hit the ground running with TAO. The notebooks install TAO, download the required data, and run TAO commands end-to-end, for various use cases.

    These notebooks are broadly classified into the following 3 categories.

    • tao_api_starter_kit: These are end-to-end notebooks that help you learn the features

      supported by the TAO Toolkit API model of execution. The notebooks under the api directory, work directly at the REST API level using REST API requests, while the client directory uses the TAO Client CLI to interact with the API server.

    • tao_launcher_starter_kit: This directory includes sample notebooks that walk you through the end-to-end workflow for all the computer vision models

      supported in TAO Toolkit. You can interact with the Toolkit using the TAO Toolkit launcher CLI.

    • tao_data_services: This directory includes sample notebooks that walk you through the end-to-end workflow of the different

      dataset manipulation and annotation tools that are included as part of the TAO Toolkit.

Copy
Copied!
            

setup |--> quickstart_launcher.sh |--> quickstart_api_bare_metal |--> quickstart_api_aws_eks |--> quickstart_api_azure_aks |--> quickstart_api_gcp_gke notebooks |--> tao_api_starter_kit |--> api |--> automl |--> end2end |--> dataset_prepare |--> client |--> automl |--> end2end |--> dataset_prepare |--> tao_launcher_starter_kit |--> dino |--> deformable_detr |--> classification_pyt |--> ocdnet |--> ... |--> tao_data_services |--> data |--> ...

TAO toolkit is available as a Docker container or a collection of Python wheels. You can run the TAO Toolkit in one of the following ways:

Launcher CLI

The TAO Toolkit Launcher is a lightweight Python based command-line interface. The launcher acts as a front-end for TAO Toolkit containers built on top of PyTorch and TensorFlow. The CLI abstracts you from the information about which network is implemented in what container. The respective container gets launched automatically based on the model you plan to use.

To get started with the launcher, use the follow the instructions to install the required pre-requisite software.

Installing the Pre-requisites

The TAO Toolkit Launcher is strictly a Python3 only package, capable of running on Python versions >= 3.7.

  1. Install docker-ce by following the official instructions.

    After you have installed docker-ce, follow the post-installation steps to ensure that the Docker can be run without sudo.

  2. Install nvidia-container-toolkit by following the install-guide.

  3. Get an NGC account and API key:

    1. Go to NGC and click the TAO Toolkit container in the Catalog tab. This message is displayed: “Sign in to access the PULL feature of this repository”.

    2. Enter your Email address and click Next, or click Create an Account.

    3. Choose your organization when prompted for Organization/Team.

    4. Click Sign In.

  4. Log in to the NGC Docker registry (nvcr.io) using the command docker login nvcr.io and enter the following credentials:

    Copy
    Copied!
                

    a. Username: "$oauthtoken" b. Password: "YOUR_NGC_API_KEY"


    where YOUR_NGC_API_KEY corresponds to the key you generated from step 3.

Note

DeepStream 6.0 - NVIDIA SDK for IVA inference is recommended.

  1. Setup your Python environment with Python version >= 3.7

    Note

    NVIDIA recommends setting up a Python environment using miniconda. The following instructions show how to setup a Python conda environment.

    1. Follow the instructions in this link to set up a Conda environment using Miniconda.

    2. After you have installed miniconda, create a new environment by setting the Python version to 3.7.

      Copy
      Copied!
                  

      conda create -n launcher python=3.7

    3. Activate the conda environment that you have just created.

      Copy
      Copied!
                  

      conda activate launcher

    4. Once you have activated your conda environment, verify that the command prompt shows the name of your Conda environment.

      Copy
      Copied!
                  

      (launcher) py-3.7 desktop:

    5. When you are done with your session, you can deactivate your conda environment using the deactivate command:

      Copy
      Copied!
                  

      conda deactivate

    6. You may re-instantiate this conda environment using the following command.

    Copy
    Copied!
                

    conda activate launcher


Installing TAO Launcher

After you have installed the required pre-requisites:

  1. Install the CLI launcher using the quick start script downloaded with the getting_started NGC package from here

    Copy
    Copied!
                

    bash setup/quickstart_launcher.sh --install

  2. Use this script to update the launcher to the latest version of TAO Toolkit by running the following command:

    Copy
    Copied!
                

    bash setup/quickstart_launcher.sh --upgrade

  3. Invoke the entry points using the tao command.

    Copy
    Copied!
                

    tao --help

    The sample output of the above command is:

    Copy
    Copied!
                

    usage: tao [-h] {list,stop,info,dataset,deploy,model} ... Launcher for TAO optional arguments: -h, --help show this help message and exit task_groups: {list,stop,info,dataset,deploy,model}

    Under tasks you can see all the launcher-invokable tasks. The following are the specific tasks that help with handling the launched commands using the TAO Launcher:

    • list

    • stop

    • info

Note

When installing the TAO Toolkit Launcher to your host machine’s native Python3 as opposed to the recommended route of using virtual environment, you may get an error saying that tao binary wasn’t found. This is because the path to your tao binary installed by pip wasn’t added to the PATH environment variable in your local machine. To fix this, run the following command:

Copy
Copied!
            

export PATH=$PATH:~/.local/bin

Running from Container

You can run TAO directly using the Docker container. To use the container directly, you need to know which container to pull. There are multiple containers under TAO, and depending on the model that you want to train you must pull the appropriate container. This is not required when using the Launcher CLI.

Copy
Copied!
            

export DOCKER_REGISTRY="nvcr.io" export DOCKER_NAME="nvidia/tao/tao-toolkit" export DOCKER_TAG="5.0.0-tf1.15.5" ## for TensorFlow docker export DOCKER_CONTAINER=$DOCKER_REGISTRY/$DOCKER_NAME:$DOCKER_TAG docker run -it --rm --gpus all -v /path/in/host:/path/in/docker $DOCKER_CONTAINER \ detectnet_v2 train -e /path/to/experiment/spec.txt -r /path/to/results/dir -k $KEY --gpus 4

For detailed instructions on how to run directly from containers, refer to this section.

Running TAO Toolkit APIs

TAO Toolkit API is a Kubernetes service that enables building end-to-end AI models using REST APIs. The API service can be installed on a Kubernetes cluster (local / AWS EKS) using a Helm chart along with minimal dependencies. TAO Toolkit jobs can be run using the GPUs available on the cluster and can scale to a multi-node setting. You can use a TAO client CLI to interact with TAO services remotely or you can integrate it with your apps and services directly using REST APIs.

To get started, use the provided one-click deploy script to deploy either on bare-metal setup or on managed Kubernetes service like Amazon EKS. Jupyter Notebooks to train using the APIs directly or using the client app is provided under notebooks/api_starter_kit.

Copy
Copied!
            

bash setup/quickstart_api_bare_metal/setup.sh install bash setup/quickstart_api_aws_eks/setup.sh install

More information about setting up the API services and the API is provided

here.

Running from Python Wheels

You can also run TAO directly on bare-metal without Docker or K8s by using the Python wheels. These Python wheels contain stand alone implementations of the DNN functionality that are pre-built and packaged into the TAO Toolkit containers.

The table below helps map the TAO Toolkit wheel to the container and captures any exceptions about these wheels.

Wheel Name

Container Mapping

Networks Supported

nvidia-tao-pytorch nvcr.io/nvidia/tao/tao-toolkit:5.3.0-pytorch
  • action_recognition
  • centerpose
  • mal
  • ml_recog
  • ocdnet
  • ocrnet
  • optical_inspection
  • pose_classification
  • re_identification
  • classification_pyt
  • segformer
nvidia-tao-deploy nvcr.io/nvidia/tao/tao-toolkit:5.3.0-deploy
  • visual_changenet
  • centerpose
  • classification_pyt
  • classification_tf1
  • classification_tf2
  • deformable_detr
  • detectnet_v2
  • dino
  • dssd
  • efficientdet_tf1
  • efficientdet_tf2
  • faster_rcnn
  • lprnet
  • mask_rcnn
  • ml_recog
  • multitask_classification
  • ocdnet
  • ocrnet
  • optical_inspection
  • retinanet
  • segformer
  • ssd
  • trtexec
  • unet
  • yolo_v3
  • yolo_v4
  • yolo_v4_tin

TAO Toolkit provides sample tutorials to deploy directly on Google Colab without having to configure your infrastructure. The full instructions on how to work with Google Colab are provided here.

Installing nvidia_tao_deploy locally

The section below details the instructions on how to install the nvidia_tao_deploy wheels locally

  1. Install the following Python pip dependencies:

    Copy
    Copied!
                

    python3 -m pip install --upgrade pip python3 -m pip install Cython==0.29.36 python3 -m pip install nvidia-ml-py python3 -m pip install nvidia-pyindex python3 -m pip install --upgrade setuptools python3 -m pip install pycuda==2020.1 python3 -m pip install nvidia-eff-tao-encryption python3 -m pip install nvidia-eff python3 -m pip install cffi

  2. Setup openMPI and mpi4py:

    Copy
    Copied!
                

    sudo apt-get install libopenmpi-dev -y python3 -m pip install mpi4py

  3. Install the TAO wheel:

    Copy
    Copied!
                

    python3 -m pip install nvidia-tao-deploy

Installing nvidia_tao_pytorch Locally

The nvidia-tao-pytorch wheel has several 3rd party dependencies, which can be quite cumbersome to install. Inorder to help with the build installation, please follow the steps in this script:

As a part of the TAO Toolkit package, example Jupyter Notebooks are available for all the tasks that are supported in TAO. The Notebooks broadly cover the following steps:

  1. Setting up the local enviroment for TAO.

  2. Installing the TAO Toolkit launcher CLI.

  3. Download the dataset and pre-trained model.

  4. Running the end-to-end workflow of the computer vision task. A brief description of the Notebook is covered in the tables below.

Computer Vision

All computer vision samples are included in the getting started resource on NGC.

Purpose-Built Pre-Trained Models

The following is a list of purpose-built pre-trained models mapped with their corresponding samples.

Model Name

Jupyter Notebook

Description

VehicleTypeNet notebooks/tao_launcher_starter_kit/classification/classification.ipynb Sample notebook to train, prune, and optimize a resnet18 image classification model on Cats and Dogs dataset
VehicleMakeNet notebooks/tao_launcher_starter_kit/classification/classification.ipynb Sample notebook to train, prune, and optimize a resnet18 image classification model on Cats and Dogs dataset
TrafficCamNet notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb Sample notebook to train, prune, and optimize a resnet18 DetectNet-v2 object detection model on KITTI dataset
PeopleSegNet notebooks/tao_launcher_starter_kit/mask_rcnn/mask_rcnn.ipynb Sample notebook to train, prune, and optimize a MaskRCNN instance segmentation model on MS-COCO dataset
PeopleNet notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb Sample notebook to train, prune, and optimize a resnet18 DetectNet-v2 object detection model on KITTI dataset
License Plate Recognition notebooks/tao_launcher_starter_kit/lprnet/lprnet.ipynb Sample notebook to train and optimize a License Plate Recognition model
License Plate Detection notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb Sample notebook to train, prune, and optimize a resnet18 DetectNet-v2 object detection model on KITTI dataset
Facial Landmark notebooks/tao_launcher_starter_kit/fpenet/fpenet.ipynb Sample notebook to train and optimize a Fiducial Point estimation network based on AFW dataset
FaceDetectIR notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb Sample notebook to train, prune, and optimize a resnet18 DetectNet-v2 object detection model on KITTI dataset
FaceDetect notebooks/tao_launcher_starter_kit/facenet/facenet.ipynb Sample notebook to train, prune, and optimize a resnet18 DetectNet-v2 object detection model on WIDER Faces dataset
DashCamNet notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb Sample notebook to train, prune, and optimize a resnet18 DetectNet-v2 object detection model on KITTI dataset
BodyPoseNet notebooks/tao_launcher_starter_kit/bpnet/bpnet.ipynb Sample notebook to train, prune, and optimize a resnet18 BodyPoseNet model on COCO dataset
ActionRecognitionNet notebooks/tao_launcher_starter_kit/actionrecognitionnet/actionrecognitionnet.ipynb Sample notebook to train and optimize an Action Recognition model on HMDB51 dataset
PoseClassificationNet notebooks/tao_launcher_starter_kit/pose_classification_net/poseclassificationnet.ipynb Sample notebook to train and optimize a Pose Classification network on Market-1501 dataset
PointPillars notebooks/tao_launcher_starter_kit/pointpillars/pointpillars.ipynb Sample notebook to train, prune, and optimize a 3-D Object Detection model on KITTI point cloud dataset
ReIdentificationNet notebooks/tao_launcher_starter_kit/re_identification_net/reidentificationnet_resnet.ipynb Sample notebook to train and optimize a Re-Identification network on Market-1501 dataset
ReIdentificationNet Transformer notebooks/tao_launcher_starter_kit/re_identification_net/reidentificationnet_swin.ipynb Sample notebook to train and optimize a Re-Identification Transformer network on Market-1501 dataset
OCDNet notebooks/tao_launcher_starter_kit/ocdnet/ocdnet.ipynb Sample notebook to train, prune, and optimize an optical character detection model on the ICDAR2015 dataset
OCRNet notebooks/tao_launcher_starter_kit/ocrnet/ocrnet.ipynb Sample notebook to train, prune, and optimize an optical character recognition model on the ICDAR2015 dataset
Optical Inspection notebooks/tao_launcher_starter_kit/optical_inspection/OpticalInspection.ipynb Sample notebook to train and optimize a siamese model for optical inspection of PCB components on a custom dataset
Retail object recognition notebooks/tao_launcher_starter_kit/metric_learning_recogntition/metric_learning_recogntition.ipynb Sample notebook to train and optimize a metric learning recognition model on Retail Product Checkout Dataset
VisualChangeNet-Classification notebooks/tao_launcher_starter_kit/visual_changenet/visual_changenet_classification.ipynb Sample notebook to train and optimize a visual changenet model for optical inspection of PCB components on a custom dataset
CenterPose notebooks/tao_launcher_starter_kit/centerpose/centerpose.ipynb Sample notebook to train and optimize a centerpose model for estimating the object pose on Google Objectron dataset

Open model architecture

Network Architecture

Jupyter Notebook

Description

DetectNet_v2 notebooks/tao_launcher_starter_kit/detectnet_v2/detectnet_v2.ipynb Sample notebook to train, prune, and optimize a resnet18 DetectNet-v2 object detection model on KITTI dataset
FasterRCNN notebooks/tao_launcher_starter_kit/faster_rcnn/faster_rcnn.ipynb Sample notebook to train, prune, and optimize a resnet18 FasterRCNN object detection model on KITTI dataset
YOLOv3 notebooks/tao_launcher_starter_kit/yolo_v3/yolo_v3.ipynb Sample notebook to train, prune, and optimize a resnet18 YOLOv3 object detection model on KITTI dataset
YOLOv4 notebooks/tao_launcher_starter_kit/yolo_v4/yolo_v4.ipynb Sample notebook to train, prune, and optimize a resnet18 YOLOv4 object detection model on KITTI dataset
YOLOv4-tiny notebooks/tao_launcher_starter_kit/yolo_v4_tiny/yolo_v4_tiny.ipynb Sample notebook to train, prune, and optimize a YOLOv4-Tiny object detection model on KITTI dataset
SSD notebooks/tao_launcher_starter_kit/ssd/ssd.ipynb Sample notebook to train, prune, and optimize a resnet18 SSD object detection model on KITTI dataset
DSSD notebooks/tao_launcher_starter_kit/dssd/dssd.ipynb Sample notebook to train, prune, and optimize a resnet18 DSSD object detection model on KITTI dataset
RetinaNet notebooks/tao_launcher_starter_kit/retinanet/retinanet.ipynb Sample notebook to train, prune, and optimize a resnet18 RetinaNet object detection model on KITTI dataset
MaskRCNN notebooks/tao_launcher_starter_kit/mask_rcnn/mask_rcnn.ipynb Sample notebook to train, prune, and optimize a resnet18 MaskRCNN instance segmentation model on COCO dataset
UNET notebooks/tao_launcher_starter_kit/unet/unet_isbi.ipynb Sample notebook to train, prune, and optimize a Vanilla UNet semantic segmentation model on ISBI dataset
Classification (TF1) notebooks/tao_launcher_starter_kit/classification_tf1/classification.ipynb Sample notebook to train, prune, and optimize a resnet18 image classification model on Cats and Dogs dataset
Classification (TF2) notebooks/tao_launcher_starter_kit/classification_tf2/classification.ipynb Sample notebook to train, prune, and optimize a EfficientNet-b0 image classification model on Cats and Dogs dataset
EfficientDet (TF1) notebooks/tao_launcher_starter_kit/efficientdet_tf1/efficientdet.ipynb Sample notebook to train, prune, and optimize a EfficientDet-D0 object detection model on COCO dataset
EfficientDet (TF2) notebooks/tao_launcher_starter_kit/efficientdet_tf2/efficientdet.ipynb Sample notebook to train, prune, and optimize a EfficientDet-D0 object detection model on COCO dataset
PointPillars notebooks/tao_launcher_starter_kit/pointpillars/pointpillars.ipynb Sample notebook to train, prune, and optimize a 3-D Object Detection model on KITTI point cloud dataset
Deformable DETR notebooks/tao_launcher_starter_kit/deformable_detr/deformable_detr.ipynb Sample notebook to train and optimize a ResNet-50 Deformable-DETR model on COCO dataset
DINO notebooks/tao_launcher_starter_kit/dino/dino.ipynb Sample notebook to train and optimize a ResNet-50 DINO model on COCO dataset
SegFormer notebooks/tao_launcher_starter_kit/segformer/segformer.ipynb Sample notebook to train and optimize a MIT-B5 SegFormer semantic segmentation model on ISBI dataset
Classification (PyT) notebooks/tao_launcher_starter_kit/classification_pyt/classification.ipynb Sample notebook to train and optimize a FAN based image classification model on Cats/Dogs dataset
VisualChangeNet-Segmentation notebooks/tao_launcher_starter_kit/visual_changenet/visual_changenet_segmentation.ipynb Sample notebook to train and optimize a visual changenet model on LEVIR-CD dataset for segmentation change detection
CenterPose notebooks/tao_launcher_starter_kit/centerpose/centerpose.ipynb Sample notebook to train and optimize a centerpose model on Google Objectron dataset for object pose estimation

The TAO Toolkit Docker gives you access to a repository of pretrained models that can serve as a starting point when training deep neural networks. These models are hosted on the NGC. To download the models:

  1. Download the NGC CLI and install it. More information about the NGC Catalog CLI is available here.

  2. Follow the instructions below to configure the NGC CLI and download the models.

Listing all Available Models

Use this command to get a list of models that are hosted in the NGC model registry:

Copy
Copied!
            

ngc registry model list <model_glob_string>

Here is an example of using this command for the computer vision models:

Copy
Copied!
            

ngc registry model list nvidia/tao/pretrained_*

This command gives us a list of the pretrained backbones available for different tasks:

  • Classification

  • Object Detection with Detectnet_v2

  • Object Detection with SSD/DSSD/YOLOv3/YOLOv4/YOLOv4-Tiny/FasterRCNN/RetinaNet

  • Object Detection with EfficientDet

  • Instance Segmentation

  • Semantic Segmentation

Note

All our classification models have names based on this template: nvidia/tao/pretrained_classification:<template>.

To view the full list of computer vision models, use the following command:

Copy
Copied!
            

ngc registry model list nvidia/tao/*

Downloading a Model

Use this command to download the model you have chosen from the NGC model registry:

Copy
Copied!
            

ngc registry model download-version <org/team/model_name:version> -dest <path_to_download_dir>

For example, use this command to download the resnet 18 classification model to the $USER_EXPERIMENT_DIR directory:

Copy
Copied!
            

ngc registry model download-version nvidia/tao/pretrained_classification:resnet18 --dest $USER_EXPERIMENT_DIR/pretrained_resnet18

Copy
Copied!
            

Downloaded 82.41 MB in 9s, Download speed: 9.14 MB/s ---------------------------------------------------- Transfer id: pretrained_classification_vresnet18 Download status: Completed. Downloaded local path: /workspace/tao-experiments/pretrained_resnet18/ Total files downloaded: 2 Total downloaded size: 82.41 MB Started at: 2019-07-16 01:29:53.028400 Completed at: 2019-07-16 01:30:02.053016 Duration taken: 9s seconds

Once you have downloaded the notebook samples and required pretrained models, you can start the respective sample notebook with the following command:

Copy
Copied!
            

jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root

Open an internet browser on localhost and navigate to the following URL:

Copy
Copied!
            

http://0.0.0.0:8888

Note

If you want to run the notebook from a remote server, follow these steps.

Execute the cells in the notebook to train a model using TAO Toolkit.

After training is complete, follow these instructions to deploy a computer vision model to DeepStream.

Previous Overview
Next TAO Toolkit Launcher
© Copyright 2023, NVIDIA.. Last updated on Aug 26, 2024.