NVIDIA TAO Toolkit v30.2202

Introduction

• Overview
  
  • Pre-trained Models
  • TAO Computer Vision Workflow Overview
  • TAO Conversational AI Workflow Overview
• TAO Toolkit Quick Start Guide
  
  • Requirements
  • Training with Jupyter Notebook
• TAO Toolkit Launcher
  
  • Running the launcher
  • Handling launched processes
  • Useful Environment variables
  • Invoking the containers directly
• Migrating to TAO Toolkit

CV Model Zoo

• PeopleNet
  
  • Training algorithm
  • Intended use case
• TrafficCamNet
  
  • Training algorithm
  • Intended Use Case
• DashCamNet
  
  • Training algorithm
  • Intended use case
• LPDNet
  
  • Training Algorithm
  • Intended Use Case
• LPRNet
  
  • Training algorithm
  • Intended use case
• VehicleTypeNet
  
  • Training Algorithm
  • Intended Use
• VehicleMakeNet
  
  • Training Algorithm
  • Intended Use
• PeopleSegNet
  
  • Training Algorithm
  • Intended Use
• PeopleSemSegNet
  
  • Training Algorithm
  • Intended Use
• FaceDetect-IR
  
  • Training algorithm
  • Intended use case
• FaceDetect
  
  • Training algorithm
  • Intended use case
• Gaze Estimation
  
  • Training algorithm
  • Intended use case
• Emotion Classification
  
  • Training algorithm
  • Intended use case
• HeartRate Estimation
  
  • Training algorithm
  • Intended use case
• Facial Landmarks Estimation
  
  • Model Architecture
  • Training Algorithm
  • References
  • Intended Use
• Gesture Recognition
  
  • Model Overview
  • Model Architecture
  • Training Algorithm
  • Reference
  • Intended Use
• Body Pose Estimation
  
  • Model Architecture
  • Training algorithm
  • Reference
  • Intended use case
• Open Images
  
  • Overview
  • Open Images Pre-trained Image Classification
  • Open Images Pre-trained Object Detection
  • Open Images Pre-trained DetectNet_v2
  • Open Images Pre-trained EfficientDet
  • Open Images Pre-trained Instance Segmentation
  • Open Images Pre-trained Semantic Segmentation

Conv AI Model Zoo

• Conversational AI Model Zoo

Running TAO Toolkit in the Cloud

• Running TAO Toolkit in the Cloud
• Running TAO Toolkit on an AWS VM
  
  • Pre-Requisites
  • Setting up an AWS EC2 instance
  • Installing the Pre-Requisites for TAO Toolkit in the VM
  • Download and run the test samples
• Running TAO Toolkit on Google Cloud Platform
  
  • Setting up a VM Linux VM Instance
  • Using the VM
  • Setting up the VM and Enabling GPUs
  • Installing the Pre-requisites for TAO Toolkit
  • Downloading and Running Test Samples
• Running TAO Toolkit on an Azure VM
  
  • Setting up an Azure VM
  • Installing the Pre-Requisites for TAO Toolkit in the VM
  • Downloading and Running the Test Samples

CV Applications

• Offline Data Augmentation
  
  • Object Detection
• Optimizing the Training Pipeline
  
  • Quantization Aware Training
  • Automatic Mixed Precision
• Data Annotation Format
  
  • Image Classification Format
  • Object Detection – KITTI Format
  • Object Detection – COCO Format
  • Instance Segmentation – COCO format
  • Semantic Segmentation – UNet Format
  • Gesture Recognition – Custom Format
  • Heart Rate Estimation – Custom Format
  • EmotionNet, FPENET, GazeNet – JSON Label Data Format
  • BodyposeNet – COCO Format
• Image Classification
  
  • Preparing the Input Data Structure
  • Creating an Experiment Spec File - Specification File for Classification
  • Training the model
  • Evaluating the Model
  • Running Inference on a Model
  • Pruning the Model
  • Re-training the Pruned Model
  • Exporting the model
  • Deploying to DeepStream
• Object Detection
  
  • DetectNet_v2
  • FasterRCNN
  • YOLOv3
  • YOLOv4
  • YOLOv4-tiny
  • SSD
  • DSSD
  • RetinaNet
  • EfficientDet
• Instance Segmentation
  
  • Data Input for Instance Segmentation
  • MaskRCNN
• Semantic Segmentation
  
  • UNET
• Gaze Estimation
  
  • Gaze Estimation
• Emotion Classification
  
  • Emotion Classification
• HeartRate Estimation
  
  • Heart Rate Estimation
• Facial Landmarks Estimation
  
  • Facial Landmarks Estimation
• Gesture Recognition
  
  • Gesture Recognition
• Body Pose Estimation
  
  • Body Pose Estimation
• Multitask Image Classification
  
  • Preparing the Input Data Structure
  • Creating an Experiment Spec File - Specification File for Multitask Classification
  • Training the model
  • Evaluating the Model
  • Generating Confusion Matrix
  • Running Inference on a Model
  • Pruning the Model
  • Re-training the Pruned Model
  • Exporting the model
  • Deploying to DeepStream
• Character Recognition
  
  • LPRNet
• ActionRecognitionNet
  
  • Preparing the Dataset
  • Creating an Experiment Spec File
  • Training the Model
  • Evaluating the model
  • Running Inference on the Model
  • Exporting the Model
  • Deploying the Model

Conversational AI Applications

• ASR
  
  • Speech Recognition
  • Speech Recognition With CitriNet
• Natural Language Processing
  
  • Joint Intent and Slot Classification
  • Punctuation and Capitalization
  • Question Answering
  • Text Classification
  • Token Classification (Named Entity Recognition)
• Language Models
  
  • N-Gram Language Model
• Speech Synthesis or Text To Speech
  
  • Overview
  • Spectrogram Generator
  • Vocoder

Deploying to Inference SDKs

• Integrating TAO CV Models with Triton Inference Server
• TensorRT
  
  • TensorRT Open Source Software
  • Installing the TAO Converter
  • Running the TAO converter
• Integrating Conversational AI Models into Riva
• Integrating TAO Models into DeepStream
  
  • Installation Prerequisites
  • Deployment Files
  • Sample Application

More Information

• Release Notes
  
  • Version list
  • TAO Toolkit 3.0-22.02
  • TAO Toolkit 3.0-21.11
  • TAO Toolkit 3.0-21.08
• Frequently Asked Questions
  
  • TLT to TAO Toolkit rename
  • Model support
  • Pruning
  • Model Export and Deployment
  • Training
• Troubleshooting Guide
  
  • NGC
  • TAO Launcher
  • MaskRCNN
  • DetectNet_v2
  • Natural Language Processing
• Support Information
• Acknowledgements
  
  • nitime
  • OpenSSL
  • JsonCpp
  • Python
  • libcurl
  • OpenCV
  • zlib
  • TensorFlow
  • Keras
  • PyTorch
  • ssd_keras
  • Yamale
  • PyCUDA
  • protobuf
  • onnx
  • PIL
  • PyYAML
  • addict
  • argcomplete
  • bto3
  • cryptography
  • docker
  • dockerpty
  • gRPC
  • h5py
  • jupyter
  • numba
  • numpy
  • pandas
  • posix_ipc
  • prettytable
  • arrow
  • PyJWT
  • requests
  • retrying
  • seaborn
  • scikit-image
  • scikit-learn
  • semver
  • Shapely
  • simplejson
  • six
  • python-tabulate
  • toposort
  • tqdm
  • uplink
  • xmltodict
  • recordclass
  • cocoapi
  • mpi4py
  • Open MPI
  • lazy_object_proxy
  • onnxruntime
  • pytorch-lightning
  • KenLM
  • Eigen
  • google/automl