Jetson Linux API Reference

32.7.3 Release
frontend (TensorRT multichannel video capture)

Requirements

  • Camera
  • Display


Overview

This sample demonstrates video capture for four video streams coming directly from camera sensors, where the streams have different resolutions. Three of the streams go through the encoding process and are saved as H.264 or H.265 files. The fourth stream provides full resolution and uses NVIDIA® TensorRT to perform object detection. If target object is detected, it will draw a bounding box around the object on the frame. Finally, the frame with the bounding box is rendered on the display and simultaneously encoded as an H.264 or H.265 file.


Building and Running

Prerequisites

To install TensorRT

  1. Open the apt source configuration file in a text editor:
      $ sudo vi /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
    
  2. Change the repository name and download URL in the deb commands shown below:

      deb https://repo.download.nvidia.com/jetson/common <release> main
      deb https://repo.download.nvidia.com/jetson/<platform> <release> main
    

    <release>> is the release number. Ex: r32.5.

    <platform> identifies the platform's processor.

    • t194 for Jetson AGX Xavier series or Jetson Xavier NX
    • t186 for Jetson TX2 series
    • t210 for Jetson Nano or Jetson TX1
  3. Enter:
      $ sudo apt-get update
      $ sudo apt-get install tensorrt
    

To build

  1. Ensure the following variable is set to 1 by making the following change in the Makefile:
      ENABLE_TRT := 1
    
  2. Enter:
    $ cd frontend
    $ make
    

To run

  • Enter:
     $ ./frontend --deploy ../../data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.prototxt \
         --model ../../data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.caffemodel \
         --no-preview
    

To quit

  • Enter q.

To view command-line options:

  • Enter:
     $ cd frontend
     $ ./frontend -h
    


Flow

The following diagrams illustrate the flow of data through this sample.

Detailed Buffer Management

Detailed Buffer Management

Buffer Flow of the Frontend Using TensorRT

Buffer Flow of the Frontend Using TensorRT

   $ ./frontend --deploy ../../data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.prototxt \
       --model ../../data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.caffemodel


Key Classes and Methods

For information on libargus, see Libargus Camera API. For information on EGLStream, see documentation on the Khronos web site.

StreamConsumer Class

The sample defines StreamConsumer as an abstracted class. It is not imported from other libraries.

Class Description
StreamConsumer A base class of the libargus OutputStream consumer. It contains a dedicated thread to poll frames from Argus::OutputStream.

The following table describes key StreamConsumer members.

Members Description
setOutputStream Sets OutputStream which be used to create the FrameConsumer.
getSize Gets stream resolution.
threadInitialize Virtual method that initializes the consumer thread.
threadExecute Virtual method that manages the thread loop.
threadShutdown Virtual method that destroys the consumer thread.
processFrame Virtual method that handles one frame.

VideoEncoder Class

The sample defines VideoEncoder as an abstracted class. It is not imported from other libraries.

Class Description
VideoEncoder This is a utility class to simplify the use of V4L2 video encoder.

The following table describes key VideoEncoder members.

Members Description
VideoEncoder Constructor in which to specify resolution, output video filename, and select encoding format.
initialize Initialize video encoder and stream consumer.
shutdown Destroy resources.
setBufferDoneCallback Set callback to return buffer to caller.
encodeFromFd Feed the encoder with a new buffer.

VideoStreamConsumer Class

The sample defines VideoStreamConsumer as an abstracted class. It is not imported from other libraries.

Class Description
VideoStreamConsumer This class is a derived class of StreamConsumer to support video encoding. It reuses the VideoEncoder object for video encoding.

The following table describes key VideoStreamConsumer members.

Members Description
VideoEncodeStreamConsumer Constructor in which to specify the name of the consumer, output video filename and select encoding format.
threadInitialize Initializes the video encoder and stream consumer.
threadShutdown Destroys resources.
processFrame Receives frames and pushes them to the encoder.

TRTStreamConsumer Class

The sample defines TRTStreamConsumer as an abstracted class. It is not imported from other libraries.

Class Description
TRTStreamConsumer This class is a derived class of StreamConsumer to support TensorRT (GIE) inference. It also has a VideoEncoder object to support video encoding.

The following table describes key TRTStreamConsumer members.

Members Description
TRTStreamConsumer Constructor in which to specify the name of the consumer, output video filename and select encoding format.
threadInitialize Initializes TensorRT context, video encoder and stream consumer.
threadShutdown Destroys resources.
processFrame Receives frames and prepares them for the TensorRT inference.
setDeployFile Sets the deploy file for building the TensorRT context.
setModelFile Sets the model file for building the TensorRT context.
RenderThreadProc Loops to draw the bounding box, render the frame, and encode the video.
TRTThreadProc Loops to do the TensorRT inference.