Jetson Linux API Reference
36.2 Release
02_video_dec_cuda (CUDA processing with decode)

Overview

This sample demonstrates how the buffer allocated by the libv4l2 component is used to decode H.265/H.264 video streams, and how CUDA is used aid in rendering images without extra memory copy operations.

Supported video formats:

  • H.264
  • H.265

Building and Running

Prerequisites

To build:

  • Enter: 
     $ cd /usr/src/jetson_multimedia_api/samples/02_video_dec_cuda
 $ make

To run

  • Enter: 
     $ ./video_dec_cuda <in-file> <in-format> [options]

Example

   $ ./video_dec_cuda ../../data/Video/sample_outdoor_car_1080p_10fps.h264 H264

Flow

The following diagram shows the flow of data through the sample.

  • The Output Plane receives input in bitstream format and delivers it to the NvVideoDecoder for decoding.
  • Decoded frames are transferred to the Capture Plane in YUV format.
  • If OSD is enabled, the output format of VIC Transform will be RGBA. the OSD will put text which user defined onto the video.
  • The CUDA Process calls CUDA kernel to draw a black box with 32x32 pixels at the top left side of each frame
  • The application can also dump YUV files from the VIC Transform.
  • The application transfers the decoded YUV frames to post processor and then to EglRenderer for rendering.

Additional Details

The main flow of this sample is as follows:

Tegra provides pitch linear and block linear memory format support on input and output. Input and output memory formats do not need to match. 

    ret = NvBufSurf::NvTransform(&transform_params,
                                              dec_buffer->planes[0].fd,
                                              ctx->dst_dma_fd);

This call converts the video format and the layout, and application can set the output format of the conversion.

NvBufSurfaceFromFd() returns the NvBufSurface pointer from the file descriptor. NvBufSurfaceMapEglImage() will create the EGLImage for the NvBufSurface. The EGLImage buffer is then used by CUDA to render a black box on the display. This is implemented in a separate common routine using EGLImage.

Key Structure and Classes

This sample uses the following key structures and classes:

Element Description
NvVideoDecoder Contains all video decoding-related elements and functions.
NvEglRenderer Contains all EGL display rendering-related functions.
egl_image The EGLImage used for CUDA processing.
dec_capture_loop The thread handler for decoding capture loop.

NvVideoDecoder packages all video decoding-related elements and functions. Key members used in the sample are:

Member Description
output_plane V4l2 output plane.
capture_plane V4l2 capture plane.
createVideoDecoder Static function to create video decode object.
subscribeEvent Subscribe event.
setExtControls Set external control to V4l2 device.
setOutputPlaneFormat Set output plane format.
setCapturePlaneFormat Set capture plane format.
getControl TBD
dqEvent Dqueue the event which report by V4l2 device.
isInError Check if under error state.

NvVideoDecoder contains two key elements: output_plane and capture_plane. These 2 objects belong to the same class type NvV4l2ElementPlane. Key members used in the sample are:

Element Description
setupPlane Setup the plane of V4l2 element.
deinitPlane Destroy the plane of V4l2 element.
setStreamStatus Start/Stop the stream.
setDQThreadCallback Set the callback function of dqueue buffer thread.
startDQThread Start the thread of dqueue buffer.
stopDQThread Stop the thread of dqueue buffer.
qBuffer Queue V4l2 buffer.
dqBuffer Dqueue V4l2 buffer.
getNumBuffers Get the number of V4l2 buffer.
getNumQueuedBuffers Get the number of V4l2 buffer which under queue.
getNthBuffer TBD

Two global functions are used to create and destroy EGLImage from the dmabuf file descriptor:

Function Description
NvBufSurfaceFromFd Get NvBufSurface from the dmabuf file descriptor.
NvBufSurfaceMapEglImage Creates the EGLImage from the NvBufSurface.
NvBufSurfaceUnMapEglImage Destroys the EGLImage.