Jetson Linux API Reference

32.7.4 Release
02_video_dec_cuda (CUDA processing with decode)

Overview

This sample demonstrates how the buffer allocated by the libv4l2 component is used to decode H.265/H.264 video streams, and how CUDA is used aid in rendering images without extra memory copy operations.

Supported video formats:

  • H.264
  • H.265


Building and Running

Prerequisites

To build:

  • Enter:
     $ cd /usr/src/jetson_multimedia_api/samples/02_video_dec_cuda
     $ make
    

To run

  • Enter:
     $ ./video_dec_cuda <in-file> <in-format> [options]
    

Example

   $ ./video_dec_cuda ../../data/Video/sample_outdoor_car_1080p_10fps.h264 H264


Flow

The following diagram shows the flow of data through the sample.

Additional Details

The main flow of this sample is as follows:

Tegra provides pitch linear and block linear memory format support on input and output. Input and output memory formats do not need to match.

    ret = ctx->conv->setOutputPlaneFormat(format.fmt.pix_mp.pixelformat,
                                              format.fmt.pix_mp.width,
                                              format.fmt.pix_mp.height,
                                              NvVideoConverter::LAYOUT_BLOCK);

This call sets up libv4l2 output plane, i.e., for receiving input as YUV data for decoder, in block linear format. This is the format that complies with Tegra hardware.

    ret = ctx->conv->setCapturePlaneFormat(format.fmt.pix_mp.pixelformat,
                                               format.fmt.pix_mp.width,
                                               format.fmt.pix_mp.height,
                                               NvVideoConverter::LAYOUT_PITCH);

This call sets up the libv4l2 capture plane in pitch linear format. This is the format that is common to the software implementation.

NvEGLImageFromFd is syntax error that returns an EGLImage pointer from file descriptor buffer that is allocated by by the member function of NvVideoDecoder base class. An EGLImage buffer is then used by CUDA to render a black box on the display. This is implemented in a separate common routine using EGLImage.


Key Structure and Classes

This sample uses the following key structures and classes:

Element Description
NvVideoDecoder Contains all video decoding-related elements and functions.
NvVideoConverter Contains elements and functions for video format conversion.
NvEglRenderer Contains all EGL display rendering-related functions.
egl_image The EGLImage used for CUDA processing.
conv_output_plane_buf_queue Output plane queue for video conversion.
dec_capture_loop The thread handler for decoding capture loop.

NvVideoDecoder packages all video decoding-related elements and functions. Key members used in the sample are:

Member Description
output_plane V4l2 output plane.
capture_plane V4l2 capture plane.
createVideoDecoder Static function to create video decode object.
subscribeEvent Subscribe event.
setExtControls Set external control to V4l2 device.
setOutputPlaneFormat Set output plane format.
setCapturePlaneFormat Set capture plane format.
getControl TBD
dqEvent Dqueue the event which report by V4l2 device.
isInError Check if under error state.

NvVideoConverter packages all video converting-related elements and functions. Key members used in the sample are:

Member Description
output_plane The output plane.
capture_plane The capture plane.
waitForIdle TBD
setOutputPlaneFormat Set output plane format.
setCapturePlaneFormat Set capture plane format.

Both NvVideoDecoder and NvVideoConverter contain 2 key elements: output_plane and capture_plane. These 2 objects belong to the same class type NvV4l2ElementPlane. Key members used in the sample are:

Element Description
setupPlane Setup the plane of V4l2 element.
deinitPlane Destroy the plane of V4l2 element.
setStreamStatus Start/Stop the stream.
setDQThreadCallback Set the callback function of dqueue buffer thread.
startDQThread Start the thread of dqueue buffer.
stopDQThread Stop the thread of dqueue buffer.
qBuffer Queue V4l2 buffer.
dqBuffer Dqueue V4l2 buffer.
getNumBuffers Get the number of V4l2 buffer.
getNumQueuedBuffers Get the number of V4l2 buffer which under queue.
getNthBuffer TBD

Two global functions are used to create and destroy EGLImage from the dmabuf file descriptor:

Function Description
NvEGLImageFromFd Create EGLImage from dmabuf file descriptor.
NvDestroyEGLImage Destroy the EGLImage.