Overview
========

.. _integrating_tlt_cv_models_with_the_inference_pipeline:

The Transfer Learning Toolkit (TLT) Computer Vision (CV) Inference Pipeline is a
C++ based SDK that provides APIs to build applications from inferences from
purpose-built pre-trained AI models. The underlying framework provides a foundation
to build multimodal applications. For example, the Gaze Estimation sample
application requires the combination of Face Detection and Facial Landmarks
(Fiducial Keypoints) Estimation.

.. _NVIDIA Triton Inference Server: https://developer.nvidia.com/nvidia-triton-inference-server
.. _NVIDIA Transfer Learning Toolkit Converter: https://developer.nvidia.com/transfer-learning-toolkit
.. _x86: https://ngc.nvidia.com/containers/nvidia:tlt-cv-inference-pipeline
.. _L4T: https://ngc.nvidia.com/containers/nvidia:tlt-cv-inference-pipeline-l4t


The TLT Computer Vision Inference Pipeline is made up of three key components:

- `NVIDIA Triton Inference Server`_: Hosts and serves AI models
- `NVIDIA Transfer Learning Toolkit Converter`_: Converts pre-trained TLT models into highly optimized TensorRT models.
- Inference Client `x86`_ or `L4T`_: Samples written in C++ that display usage of APIs to request Computer Vision inferences

.. _EmotionNet: https://ngc.nvidia.com/models/nvidia:tlt_emotionnet
.. _FaceNet: https://ngc.nvidia.com/models/nvidia:tlt_facenet
.. _FPENet: https://ngc.nvidia.com/models/nvidia:tlt_fpenet
.. _GazeNet: https://ngc.nvidia.com/models/nvidia:tlt_gazenet
.. _GestureNet: https://ngc.nvidia.com/models/nvidia:tlt_gesturenet
.. _HeartRateNet: https://ngc.nvidia.com/models/nvidia:tlt_heartratenet
.. _BodyPoseNet: https://ngc.nvidia.com/models/nvidia:tlt_bodyposenet

The purpose-built AI models that are supported by this Inference Pipeline are as follows:

- :ref:`Body Pose Estimation<heartratenet>` from `BodyPoseNet`_
- :ref:`Emotion Classification<emotionnet>` from `EmotionNet`_
- :ref:`Face Detection<detectnet_v2>` from `FaceNet`_
- :ref:`Facial Landmark Estimation<facial_landmarks_estimation>` from `FPENet`_
- :ref:`Gaze Estimation<gazenet>` from `GazeNet`_
- :ref:`Gesture Recognition<gesturenet>` from `GestureNet`_
- :ref:`Heart Rate Estimation<heartratenet>` from `HeartRateNet`_

.. tabularcolumns:: |>{\centering\arraybackslash}\X{1}{5}|>{\raggedleft\arraybackslash}\X{2}{5}|

.. csv-table:: Model Deployment
   :file: ../../content/tlt_cv_inference_models.csv
   :class: longtable
   :header-rows: 1
   :stub-columns: 1

Users can retrain supported TLT networks, drop the optimized TensorRT models
into the `NVIDIA Triton Inference Server`_, and build their own AI applications
and use-cases using the TLT CV API.

The deployment of the Inference Pipeline is managed by the
:ref:`TLT CV Quick Start Scripts<tlt_cv_quick_start_scripts>`, which are bash scripts
that pull/start relevant containers, compile TLT models, and start the Triton Server.