Jetson Linux API Reference

32.7.4 Release
V4L2 Video Decoder

Detailed Description

NVIDIA V4L2 Video Decoder Description and Extensions.

The video decoder device node is

/dev/nvhost-nvdec

Supported Pixel Formats

OUTPUT PLANE CAPTURE PLANE
V4L2_PIX_FMT_H264 V4L2_PIX_FMT_NV12M
V4L2_PIX_FMT_H265 V4L2_PIX_FMT_NV12M

Supported Memory Types

MEMORY OUTPUT PLANE CAPTURE PLANE
V4L2_MEMORY_MMAP Y Y
V4L2_MEMORY_DMABUF N Y
V4L2_MEMORY_USERPTR Y N

Supported Controls

Supported Events

Event Purpose
V4L2_EVENT_RESOLUTION_CHANGE Resolution of the stream has changed.

Handling Resolution Change Events

When the decoder generates a V4L2_EVENT_RESOLUTION_CHANGE event, the application calls STREAMOFF on the capture plane to tell the decoder to deallocate the current buffers by calling REQBUF with count zero, get the new capture plane format, and then proceed with setting up the buffers for the capture plane.

In case of decoder, the buffer format might differ from the display resolution. The application must use VIDIOC_G_CROP to get the display resolution.

EOS Handling

The following sequence must be followed for sending EOS and recieving EOS from the decoder.

  1. Send EOS to decoder by queueing on the output plane a buffer with bytesused = 0 for the 0th plane (v4l2_buffer.m.planes[0].bytesused = 0).
  2. Dequeues buffers on the output plane until it gets a buffer with bytesused = 0 for the 0th plane (v4l2_buffer.m.planes[0].bytesused == 0)
  3. Dequeues buffers on the capture plane until it gets a buffer with bytesused = 0 for the 0th plane.

Decoder Input Frame Metadata

Decoder supports reporting stream header parsing error info as input frame metadata. See V4L2_CID_MPEG_VIDEO_ERROR_REPORTING, V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA and v4l2_ctrl_video_metadata for more information.

Decoder Output Frame Metadata

Decoder supports reporting frame related metadata, including error reports and DPB info. See V4L2_CID_MPEG_VIDEO_ERROR_REPORTING, V4L2_CID_MPEG_VIDEODEC_METADATA and v4l2_ctrl_video_metadata for more information.

Note
Currently, V4L2 plugins do not support odd resolution.

Data Structures

struct  v4l2_ctrl_videodec_statusmetadata_
 Holds the decoder error status metadata for the frame. More...
 
struct  v4l2_ctrl_videodec_refframe_metadata_
 Holds the the frame specific metadata for a reference frame. More...
 
struct  v4l2_ctrl_videodec_currentframe_metadata_
 Holds the the frame specific metadata for the current frame. More...
 
struct  v4l2_ctrl_videodec_dpbinfometadata_
 Holds the decoder DPB info metadata. More...
 
struct  v4l2_ctrl_h264dec_bufmetadata_
 Holds H.264 specific decoder metadata for the frame. More...
 
struct  v4l2_ctrl_hevcdec_bufmetadata_
 Holds H.265 specific decoder metadata for the frame. More...
 
struct  v4l2_ctrl_videodec_inputbuf_metadata_
 Holds the video decoder input header error metadata for a frame. More...
 
struct  v4l2_ctrl_videodec_outputbuf_metadata_
 Holds the video decoder output metadata for a frame. More...
 

Macros

#define V4L2_CID_MPEG_VIDEO_DISABLE_COMPLETE_FRAME_INPUT   (V4L2_CID_MPEG_BASE+515)
 Defines the Control ID to indicate to the decoder that the input buffers do not contain complete buffers. More...
 
#define V4L2_CID_MPEG_VIDEO_DISABLE_DPB   (V4L2_CID_MPEG_BASE+516)
 Defines the Control ID to disable decoder DPB management. More...
 
#define V4L2_CID_MPEG_VIDEO_ERROR_REPORTING   (V4L2_CID_MPEG_BASE+517)
 Defines the Control ID to enable decoder error and metadata reporting. More...
 
#define V4L2_CID_MPEG_VIDEO_SKIP_FRAMES   (V4L2_CID_MPEG_BASE+518)
 Defines the Control ID to set the skip frames property of the decoder. More...
 
#define V4L2_CID_MPEG_VIDEODEC_METADATA   (V4L2_CID_MPEG_BASE+519)
 Defines the Control ID to get the decoder output metadata. More...
 
#define V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA   (V4L2_CID_MPEG_BASE+520)
 Defines the Control ID to get the decoder input header error metadata. More...
 
#define V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT   (V4L2_CID_MPEG_BASE+521)
 Defines the Control ID to check if display data is present. More...
 
#define V4L2_CID_VIDEODEC_HDR_MASTERING_DISPLAY_DATA   (V4L2_CID_MPEG_BASE+522)
 Defines the Control ID to get display data if V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT returns true. More...
 
#define V4L2_CID_MPEG_VIDEODEC_SAR_WIDTH   (V4L2_CID_MPEG_BASE+569)
 Defines the Control ID to get Sample Aspect Ratio width for decoding. More...
 
#define V4L2_CID_MPEG_VIDEODEC_SAR_HEIGHT   (V4L2_CID_MPEG_BASE+570)
 Defines the Control ID to get Sample Aspect Ratio height for decoding. More...
 
#define V4L2_CID_MPEG_VIDEOENC_DS_SEI_DATA   (V4L2_CID_MPEG_BASE+571)
 Defines the Control ID to embed the SEI data coming from upstream plugins. More...
 

Typedefs

typedef struct v4l2_ctrl_videodec_statusmetadata_ v4l2_ctrl_videodec_statusmetadata
 Holds the decoder error status metadata for the frame. More...
 
typedef struct v4l2_ctrl_videodec_refframe_metadata_ v4l2_ctrl_videodec_refframe_metadata
 Holds the the frame specific metadata for a reference frame. More...
 
typedef struct v4l2_ctrl_videodec_currentframe_metadata_ v4l2_ctrl_videodec_currentframe_metadata
 Holds the the frame specific metadata for the current frame. More...
 
typedef struct v4l2_ctrl_videodec_dpbinfometadata_ v4l2_ctrl_videodec_dpbinfometadata
 Holds the decoder DPB info metadata. More...
 
typedef struct v4l2_ctrl_h264dec_bufmetadata_ v4l2_ctrl_h264dec_bufmetadata
 Holds H.264 specific decoder metadata for the frame. More...
 
typedef struct v4l2_ctrl_hevcdec_bufmetadata_ v4l2_ctrl_hevcdec_bufmetadata
 Holds H.265 specific decoder metadata for the frame. More...
 
typedef struct v4l2_ctrl_videodec_inputbuf_metadata_ v4l2_ctrl_videodec_inputbuf_metadata
 Holds the video decoder input header error metadata for a frame. More...
 
typedef struct v4l2_ctrl_videodec_outputbuf_metadata_ v4l2_ctrl_videodec_outputbuf_metadata
 Holds the video decoder output metadata for a frame. More...
 

Enumerations

enum  v4l2_skip_frames_type {
  V4L2_SKIP_FRAMES_TYPE_NONE = 0,
  V4L2_SKIP_FRAMES_TYPE_NONREF = 1,
  V4L2_SKIP_FRAMES_TYPE_DECODE_IDR_ONLY = 2
}
 Enum v4l2_skip_frames_type, possible methods for decoder skip frames. More...
 
enum  v4l2_cuda_mem_type {
  V4L2_CUDA_MEM_TYPE_DEVICE = 0,
  V4L2_CUDA_MEM_TYPE_PINNED = 1,
  V4L2_CUDA_MEM_TYPE_UNIFIED = 2
}
 Enum v4l2_cuda_mem_type, possible methods for cuda memory tpye. More...
 
enum  v4l2_videodec_input_error_type {
  V4L2_DEC_ERROR_NONE = 0x0,
  V4L2_DEC_ERROR_SPS = 0x1,
  V4L2_DEC_ERROR_PPS = 0x2,
  V4L2_DEC_ERROR_SLICE_HDR = 0x4,
  V4L2_DEC_ERROR_MISSING_REF_FRAME = 0x8,
  V4L2_DEC_ERROR_VPS = 0x10
}
 Enum v4l2_videodec_input_error_type, possible error types for input stream. More...
 

Macro Definition Documentation

◆ V4L2_CID_MPEG_VIDEO_DISABLE_COMPLETE_FRAME_INPUT

#define V4L2_CID_MPEG_VIDEO_DISABLE_COMPLETE_FRAME_INPUT   (V4L2_CID_MPEG_BASE+515)

Defines the Control ID to indicate to the decoder that the input buffers do not contain complete buffers.

Note
This control must be set in case of frames containing multiple slices when the input buffers do not contain all the slices of the frame.

A boolean value must be supplied with this control.

Definition at line 501 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEO_DISABLE_DPB

#define V4L2_CID_MPEG_VIDEO_DISABLE_DPB   (V4L2_CID_MPEG_BASE+516)

Defines the Control ID to disable decoder DPB management.

Note
This only works for streams having a single reference frame.

A boolean value must be supplied with this control.

Attention
This control must be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 513 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEO_ERROR_REPORTING

#define V4L2_CID_MPEG_VIDEO_ERROR_REPORTING   (V4L2_CID_MPEG_BASE+517)

Defines the Control ID to enable decoder error and metadata reporting.

A boolean value must be supplied with this control.

Attention
This control must be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 523 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEO_SKIP_FRAMES

#define V4L2_CID_MPEG_VIDEO_SKIP_FRAMES   (V4L2_CID_MPEG_BASE+518)

Defines the Control ID to set the skip frames property of the decoder.

Decoder must be configured to skip certain types of frames. One v4l2_skip_frames_type must be passed.

Attention
This control must be set after setting formats on both the planes and before requesting buffers on either plane. This control ID is supported only for H264.

Definition at line 535 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA

#define V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA   (V4L2_CID_MPEG_BASE+520)

Defines the Control ID to get the decoder input header error metadata.

Note
Metadata reporting must be enabled using V4L2_CID_MPEG_VIDEO_ERROR_REPORTING IOCTL for this.

A pointer to a valid v4l2_ctrl_video_metadata structure must be supplied with this control.

Attention
This control must be read after dequeueing a buffer successfully from the output plane. The values in the structure are valid until the buffer is queued again.

Definition at line 565 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEODEC_METADATA

#define V4L2_CID_MPEG_VIDEODEC_METADATA   (V4L2_CID_MPEG_BASE+519)

Defines the Control ID to get the decoder output metadata.

Note
Metadata reporting must be enabled using V4L2_CID_MPEG_VIDEO_ERROR_REPORTING IOCTL for this.

A pointer to a valid v4l2_ctrl_video_metadata structure must be supplied with this control.

Attention
This control must be read after dequeueing a buffer successfully from the capture plane. The values in the structure are valid until the buffer is queued again.

Definition at line 550 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEODEC_SAR_HEIGHT

#define V4L2_CID_MPEG_VIDEODEC_SAR_HEIGHT   (V4L2_CID_MPEG_BASE+570)

Defines the Control ID to get Sample Aspect Ratio height for decoding.

This control returns unsigned integer of Sample Aspect Ratio height.

Attention
This control must be set after receiving V4L2_EVENT_RESOLUTION_CHANGE.

Definition at line 602 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEODEC_SAR_WIDTH

#define V4L2_CID_MPEG_VIDEODEC_SAR_WIDTH   (V4L2_CID_MPEG_BASE+569)

Defines the Control ID to get Sample Aspect Ratio width for decoding.

This control returns unsigned integer of Sample Aspect Ratio width.

Attention
This control must be set after receiving V4L2_EVENT_RESOLUTION_CHANGE.

Definition at line 592 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEOENC_DS_SEI_DATA

#define V4L2_CID_MPEG_VIDEOENC_DS_SEI_DATA   (V4L2_CID_MPEG_BASE+571)

Defines the Control ID to embed the SEI data coming from upstream plugins.

This control sets the SEI payload size and SEI data pointer for encoder.

Attention
This control must be set after receiving V4L2_EVENT_RESOLUTION_CHANGE.

Definition at line 612 of file v4l2_nv_extensions.h.

◆ V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT

#define V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT   (V4L2_CID_MPEG_BASE+521)

Defines the Control ID to check if display data is present.

This control returns true if HDR metadata is present in the stream.

Definition at line 573 of file v4l2_nv_extensions.h.

◆ V4L2_CID_VIDEODEC_HDR_MASTERING_DISPLAY_DATA

#define V4L2_CID_VIDEODEC_HDR_MASTERING_DISPLAY_DATA   (V4L2_CID_MPEG_BASE+522)

Defines the Control ID to get display data if V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT returns true.

This control returns display data such as display_primaries, white_point and display_parameter_luminance required for display module.

Definition at line 582 of file v4l2_nv_extensions.h.

Typedef Documentation

◆ v4l2_ctrl_h264dec_bufmetadata

Holds H.264 specific decoder metadata for the frame.

◆ v4l2_ctrl_hevcdec_bufmetadata

Holds H.265 specific decoder metadata for the frame.

◆ v4l2_ctrl_videodec_currentframe_metadata

Holds the the frame specific metadata for the current frame.

◆ v4l2_ctrl_videodec_dpbinfometadata

Holds the decoder DPB info metadata.

◆ v4l2_ctrl_videodec_inputbuf_metadata

Holds the video decoder input header error metadata for a frame.

◆ v4l2_ctrl_videodec_outputbuf_metadata

Holds the video decoder output metadata for a frame.

◆ v4l2_ctrl_videodec_refframe_metadata

Holds the the frame specific metadata for a reference frame.

◆ v4l2_ctrl_videodec_statusmetadata

Holds the decoder error status metadata for the frame.

Enumeration Type Documentation

◆ v4l2_cuda_mem_type

Enum v4l2_cuda_mem_type, possible methods for cuda memory tpye.

Enumerator
V4L2_CUDA_MEM_TYPE_DEVICE 

Memory type device.

V4L2_CUDA_MEM_TYPE_PINNED 

Memory type host.

V4L2_CUDA_MEM_TYPE_UNIFIED 

Memory type unified.

Definition at line 1602 of file v4l2_nv_extensions.h.

◆ v4l2_skip_frames_type

Enum v4l2_skip_frames_type, possible methods for decoder skip frames.

Enumerator
V4L2_SKIP_FRAMES_TYPE_NONE 

Do not skip any frame.

V4L2_SKIP_FRAMES_TYPE_NONREF 

Skip all non-reference frames.

V4L2_SKIP_FRAMES_TYPE_DECODE_IDR_ONLY 

Skip all frames except IDR.

Definition at line 1591 of file v4l2_nv_extensions.h.

◆ v4l2_videodec_input_error_type

Enum v4l2_videodec_input_error_type, possible error types for input stream.

Enumerator
V4L2_DEC_ERROR_NONE 

no error.

V4L2_DEC_ERROR_SPS 

sps error.

V4L2_DEC_ERROR_PPS 

pps error.

V4L2_DEC_ERROR_SLICE_HDR 

slice header error.

V4L2_DEC_ERROR_MISSING_REF_FRAME 

missing reference frame error.

V4L2_DEC_ERROR_VPS 

VPS error.

Definition at line 1613 of file v4l2_nv_extensions.h.