Integrating TAO CV Models with Triton Inference Server

The NVIDIA TAO provides users with an easy interface to generate accurate and optimized models for a number of computer vision use cases. These models are generally deployed via the DeepStream SDK or Riva pipelines.

Triton is an NVIDIA-developed inference software solution to efficiently deploy Deep Neural Networks (DNN) from several frameworks, such as TensorRT, Tensorflow, and ONNXRuntime. As part of this release, TAO now provides a reference application outlining the steps required to deploy a trained model into Triton.

Currently, TAO Triton apps repository shows reference implementations to integrate the following types of models into Triton Inference Server:

• DetectNet_v2

• Image Classification

• License Plate Recogntion

• YOLOv3

• MaskRCNN

• RetinaNet

• MultiTask Classification

• Pose Classification

For more information on how to deploy these models using Triton, please refer to the documentation and source code hosted in this GitHub repository.