Detailed Description

NVIDIA V4L2 Video Decoder Description and Extensions.

The video decoder device node is

/dev/nvhost-nvdec

Supported Pixel Formats

OUTPUT PLANE	CAPTURE PLANE	PLATFORM
V4L2_PIX_FMT_H264	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_H265	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_VP8	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_VP9	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_MPEG1	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_MPEG2	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_MPEG4	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_DIVX4	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_DIVX5	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_JPEG	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_MJPEG	V4L2_PIX_FMT_NV12M	T210, T186, T194, T234
V4L2_PIX_FMT_AV1	V4L2_PIX_FMT_NV12M	T234

Supported Memory Types

MEMORY	OUTPUT PLANE	CAPTURE PLANE
V4L2_MEMORY_MMAP	Y	Y
V4L2_MEMORY_DMABUF	N	Y
V4L2_MEMORY_USERPTR	Y	N

Supported Controls

V4L2_CID_MPEG_VIDEO_DISABLE_COMPLETE_FRAME_INPUT
V4L2_CID_MPEG_VIDEO_DISABLE_DPB
V4L2_CID_MPEG_VIDEO_ERROR_REPORTING
V4L2_CID_MPEG_VIDEO_SKIP_FRAMES
V4L2_CID_MIN_BUFFERS_FOR_CAPTURE (Get the minimum buffers to be allocated on capture plane. Read only. Valid after V4L2_EVENT_RESOLUTION_CHANGE)
V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA
V4L2_CID_MPEG_VIDEODEC_METADATA
V4L2_CID_MPEG_VIDEO_CUDA_MEM_TYPE
V4L2_CID_MPEG_VIDEO_CUDA_GPU_ID
V4L2_CID_MPEG_VIDEODEC_DROP_FRAME_INTERVAL

Supported Events

Event	Purpose
V4L2_EVENT_RESOLUTION_CHANGE	Resolution of the stream has changed.

Handling Resolution Change Events

When the decoder generates a V4L2_EVENT_RESOLUTION_CHANGE event, the application calls STREAMOFF on the capture plane to tell the decoder to deallocate the current buffers by calling REQBUF with count zero, get the new capture plane format, and then proceed with setting up the buffers for the capture plane.

In case of decoder, the buffer format might differ from the display resolution. The application must use VIDIOC_G_CROP to get the display resolution.

EOS Handling

The following sequence must be followed for sending EOS and recieving EOS from the decoder.

Send EOS to decoder by queueing on the output plane a buffer with bytesused = 0 for the 0th plane (v4l2_buffer.m.planes[0].bytesused = 0).
Dequeues buffers on the output plane until it gets a buffer with bytesused = 0 for the 0th plane (v4l2_buffer.m.planes[0].bytesused == 0)
Dequeues buffers on the capture plane until it gets a buffer with bytesused = 0 for the 0th plane.

Decoder Input Frame Metadata

Decoder supports reporting stream header parsing error info as input frame metadata. See V4L2_CID_MPEG_VIDEO_ERROR_REPORTING, V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA and v4l2_ctrl_video_metadata for more information.

Decoder Output Frame Metadata

Decoder supports reporting frame related metadata, including error reports and DPB info. See V4L2_CID_MPEG_VIDEO_ERROR_REPORTING, V4L2_CID_MPEG_VIDEODEC_METADATA and v4l2_ctrl_video_metadata for more information.

Note: Currently, V4L2 plugins do not support odd resolution.

Data Structures
struct	v4l2_ctrl_videodec_statusmetadata_
	Holds the decoder error status metadata for the frame. More...

struct	v4l2_ctrl_videodec_refframe_metadata_
	Holds the the frame specific metadata for a reference frame. More...

struct	v4l2_ctrl_videodec_currentframe_metadata_
	Holds the the frame specific metadata for the current frame. More...

struct	v4l2_ctrl_videodec_dpbinfometadata_
	Holds the decoder DPB info metadata. More...

struct	v4l2_ctrl_h264dec_bufmetadata_
	Holds H.264 specific decoder metadata for the frame. More...

struct	v4l2_ctrl_hevcdec_bufmetadata_
	Holds H.265 specific decoder metadata for the frame. More...

struct	v4l2_ctrl_videodec_inputbuf_metadata_
	Holds the video decoder input header error metadata for a frame. More...

struct	v4l2_ctrl_videodec_outputbuf_metadata_
	Holds the video decoder output metadata for a frame. More...

Macros
#define	V4L2_CID_MPEG_VIDEO_DISABLE_COMPLETE_FRAME_INPUT (V4L2_CID_MPEG_BASE+515)
	Defines the Control ID to indicate to the decoder that the input buffers do not contain complete buffers. More...

#define	V4L2_CID_MPEG_VIDEO_DISABLE_DPB (V4L2_CID_MPEG_BASE+516)
	Defines the Control ID to disable decoder DPB management. More...

#define	V4L2_CID_MPEG_VIDEO_ERROR_REPORTING (V4L2_CID_MPEG_BASE+517)
	Defines the Control ID to enable decoder error and metadata reporting. More...

#define	V4L2_CID_MPEG_VIDEO_SKIP_FRAMES (V4L2_CID_MPEG_BASE+518)
	Defines the Control ID to set the skip frames property of the decoder. More...

#define	V4L2_CID_MPEG_VIDEODEC_METADATA (V4L2_CID_MPEG_BASE+519)
	Defines the Control ID to get the decoder output metadata. More...

#define	V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA (V4L2_CID_MPEG_BASE+520)
	Defines the Control ID to get the decoder input header error metadata. More...

#define	V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT (V4L2_CID_MPEG_BASE+521)
	Defines the Control ID to check if display data is present. More...

#define	V4L2_CID_VIDEODEC_HDR_MASTERING_DISPLAY_DATA (V4L2_CID_MPEG_BASE+522)
	Defines the Control ID to get display data if V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT returns true. More...

#define	V4L2_CID_MPEG_VIDEODEC_SAR_WIDTH (V4L2_CID_MPEG_BASE+580)
	Defines the Control ID to get Sample Aspect Ratio width for decoding. More...

#define	V4L2_CID_MPEG_VIDEODEC_SAR_HEIGHT (V4L2_CID_MPEG_BASE+581)
	Defines the Control ID to get Sample Aspect Ratio height for decoding. More...

#define	V4L2_CID_MPEG_VIDEOENC_DS_SEI_DATA (V4L2_CID_MPEG_BASE+571)
	Defines the Control ID to embed the SEI data coming from upstream plugins. More...

Typedefs
typedef struct v4l2_ctrl_videodec_statusmetadata_	v4l2_ctrl_videodec_statusmetadata
	Holds the decoder error status metadata for the frame. More...

typedef struct v4l2_ctrl_videodec_refframe_metadata_	v4l2_ctrl_videodec_refframe_metadata
	Holds the the frame specific metadata for a reference frame. More...

typedef struct v4l2_ctrl_videodec_currentframe_metadata_	v4l2_ctrl_videodec_currentframe_metadata
	Holds the the frame specific metadata for the current frame. More...

typedef struct v4l2_ctrl_videodec_dpbinfometadata_	v4l2_ctrl_videodec_dpbinfometadata
	Holds the decoder DPB info metadata. More...

typedef struct v4l2_ctrl_h264dec_bufmetadata_	v4l2_ctrl_h264dec_bufmetadata
	Holds H.264 specific decoder metadata for the frame. More...

typedef struct v4l2_ctrl_hevcdec_bufmetadata_	v4l2_ctrl_hevcdec_bufmetadata
	Holds H.265 specific decoder metadata for the frame. More...

typedef struct v4l2_ctrl_videodec_inputbuf_metadata_	v4l2_ctrl_videodec_inputbuf_metadata
	Holds the video decoder input header error metadata for a frame. More...

typedef struct v4l2_ctrl_videodec_outputbuf_metadata_	v4l2_ctrl_videodec_outputbuf_metadata
	Holds the video decoder output metadata for a frame. More...

Enumerations
enum	v4l2_skip_frames_type { V4L2_SKIP_FRAMES_TYPE_NONE = 0, V4L2_SKIP_FRAMES_TYPE_NONREF = 1, V4L2_SKIP_FRAMES_TYPE_DECODE_IDR_ONLY = 2 }
	Enum v4l2_skip_frames_type, possible methods for decoder skip frames. More...

enum	v4l2_cuda_mem_type { V4L2_CUDA_MEM_TYPE_DEVICE = 0, V4L2_CUDA_MEM_TYPE_PINNED = 1, V4L2_CUDA_MEM_TYPE_UNIFIED = 2 }
	Enum v4l2_cuda_mem_type, possible methods for cuda memory tpye. More...

enum	v4l2_videodec_input_error_type { V4L2_DEC_ERROR_NONE = 0x0, V4L2_DEC_ERROR_SPS = 0x1, V4L2_DEC_ERROR_PPS = 0x2, V4L2_DEC_ERROR_SLICE_HDR = 0x4, V4L2_DEC_ERROR_MISSING_REF_FRAME = 0x8, V4L2_DEC_ERROR_VPS = 0x10 }
	Enum v4l2_videodec_input_error_type, possible error types for input stream. More...

Macro Definition Documentation

◆ V4L2_CID_MPEG_VIDEO_DISABLE_COMPLETE_FRAME_INPUT

#define V4L2_CID_MPEG_VIDEO_DISABLE_COMPLETE_FRAME_INPUT (V4L2_CID_MPEG_BASE+515)

Defines the Control ID to indicate to the decoder that the input buffers do not contain complete buffers.

Note: This control must be set in case of frames containing multiple slices when the input buffers do not contain all the slices of the frame.

A boolean value must be supplied with this control.

Definition at line 521 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEO_DISABLE_DPB

#define V4L2_CID_MPEG_VIDEO_DISABLE_DPB (V4L2_CID_MPEG_BASE+516)

Defines the Control ID to disable decoder DPB management.

Note: This only works for streams having a single reference frame.

A boolean value must be supplied with this control.

Attention: This control must be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 533 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEO_ERROR_REPORTING

#define V4L2_CID_MPEG_VIDEO_ERROR_REPORTING (V4L2_CID_MPEG_BASE+517)

Defines the Control ID to enable decoder error and metadata reporting.

A boolean value must be supplied with this control.

Attention: This control must be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 543 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEO_SKIP_FRAMES

#define V4L2_CID_MPEG_VIDEO_SKIP_FRAMES (V4L2_CID_MPEG_BASE+518)

Defines the Control ID to set the skip frames property of the decoder.

Decoder must be configured to skip certain types of frames. One v4l2_skip_frames_type must be passed.

Attention: This control must be set after setting formats on both the planes and before requesting buffers on either plane. This control ID is supported only for H264.

Definition at line 555 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA

#define V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA (V4L2_CID_MPEG_BASE+520)

Defines the Control ID to get the decoder input header error metadata.

Note: Metadata reporting must be enabled using V4L2_CID_MPEG_VIDEO_ERROR_REPORTING IOCTL for this.

A pointer to a valid v4l2_ctrl_video_metadata structure must be supplied with this control.

Attention: This control must be read after dequeueing a buffer successfully from the output plane. The values in the structure are valid until the buffer is queued again.

Definition at line 585 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEODEC_METADATA

#define V4L2_CID_MPEG_VIDEODEC_METADATA (V4L2_CID_MPEG_BASE+519)

Defines the Control ID to get the decoder output metadata.

Note: Metadata reporting must be enabled using V4L2_CID_MPEG_VIDEO_ERROR_REPORTING IOCTL for this.

A pointer to a valid v4l2_ctrl_video_metadata structure must be supplied with this control.

Attention: This control must be read after dequeueing a buffer successfully from the capture plane. The values in the structure are valid until the buffer is queued again.

Definition at line 570 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEODEC_SAR_HEIGHT

#define V4L2_CID_MPEG_VIDEODEC_SAR_HEIGHT (V4L2_CID_MPEG_BASE+581)

Defines the Control ID to get Sample Aspect Ratio height for decoding.

This control returns unsigned integer of Sample Aspect Ratio height.

Attention: This control must be set after receiving V4L2_EVENT_RESOLUTION_CHANGE.

Definition at line 622 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEODEC_SAR_WIDTH

#define V4L2_CID_MPEG_VIDEODEC_SAR_WIDTH (V4L2_CID_MPEG_BASE+580)

Defines the Control ID to get Sample Aspect Ratio width for decoding.

This control returns unsigned integer of Sample Aspect Ratio width.

Attention: This control must be set after receiving V4L2_EVENT_RESOLUTION_CHANGE.

Definition at line 612 of file v4l2_nv_extensions.h.

◆ V4L2_CID_MPEG_VIDEOENC_DS_SEI_DATA

#define V4L2_CID_MPEG_VIDEOENC_DS_SEI_DATA (V4L2_CID_MPEG_BASE+571)

Defines the Control ID to embed the SEI data coming from upstream plugins.

This control sets the SEI payload size and SEI data pointer for encoder.

Attention: This control must be set after receiving V4L2_EVENT_RESOLUTION_CHANGE.

Definition at line 632 of file v4l2_nv_extensions.h.

◆ V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT

#define V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT (V4L2_CID_MPEG_BASE+521)

Defines the Control ID to check if display data is present.

This control returns true if HDR metadata is present in the stream.

Definition at line 593 of file v4l2_nv_extensions.h.

◆ V4L2_CID_VIDEODEC_HDR_MASTERING_DISPLAY_DATA

#define V4L2_CID_VIDEODEC_HDR_MASTERING_DISPLAY_DATA (V4L2_CID_MPEG_BASE+522)

Defines the Control ID to get display data if V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT returns true.

This control returns display data such as display_primaries, white_point and display_parameter_luminance required for display module.

Definition at line 602 of file v4l2_nv_extensions.h.

Typedef Documentation

◆ v4l2_ctrl_h264dec_bufmetadata

typedef struct v4l2_ctrl_h264dec_bufmetadata_ v4l2_ctrl_h264dec_bufmetadata

Holds H.264 specific decoder metadata for the frame.

◆ v4l2_ctrl_hevcdec_bufmetadata

typedef struct v4l2_ctrl_hevcdec_bufmetadata_ v4l2_ctrl_hevcdec_bufmetadata

Holds H.265 specific decoder metadata for the frame.

◆ v4l2_ctrl_videodec_currentframe_metadata

typedef struct v4l2_ctrl_videodec_currentframe_metadata_ v4l2_ctrl_videodec_currentframe_metadata

Holds the the frame specific metadata for the current frame.

◆ v4l2_ctrl_videodec_dpbinfometadata

typedef struct v4l2_ctrl_videodec_dpbinfometadata_ v4l2_ctrl_videodec_dpbinfometadata

Holds the decoder DPB info metadata.

◆ v4l2_ctrl_videodec_inputbuf_metadata

typedef struct v4l2_ctrl_videodec_inputbuf_metadata_ v4l2_ctrl_videodec_inputbuf_metadata

Holds the video decoder input header error metadata for a frame.

◆ v4l2_ctrl_videodec_outputbuf_metadata

typedef struct v4l2_ctrl_videodec_outputbuf_metadata_ v4l2_ctrl_videodec_outputbuf_metadata

Holds the video decoder output metadata for a frame.

◆ v4l2_ctrl_videodec_refframe_metadata

typedef struct v4l2_ctrl_videodec_refframe_metadata_ v4l2_ctrl_videodec_refframe_metadata

Holds the the frame specific metadata for a reference frame.

◆ v4l2_ctrl_videodec_statusmetadata

typedef struct v4l2_ctrl_videodec_statusmetadata_ v4l2_ctrl_videodec_statusmetadata

Holds the decoder error status metadata for the frame.

Enumeration Type Documentation

◆ v4l2_cuda_mem_type

enum v4l2_cuda_mem_type

Enum v4l2_cuda_mem_type, possible methods for cuda memory tpye.

Enumerator
V4L2_CUDA_MEM_TYPE_DEVICE	Memory type device.
V4L2_CUDA_MEM_TYPE_PINNED	Memory type host.
V4L2_CUDA_MEM_TYPE_UNIFIED	Memory type unified.

Definition at line 1702 of file v4l2_nv_extensions.h.

◆ v4l2_skip_frames_type

enum v4l2_skip_frames_type

Enum v4l2_skip_frames_type, possible methods for decoder skip frames.

Enumerator
V4L2_SKIP_FRAMES_TYPE_NONE	Do not skip any frame.
V4L2_SKIP_FRAMES_TYPE_NONREF	Skip all non-reference frames.
V4L2_SKIP_FRAMES_TYPE_DECODE_IDR_ONLY	Skip all frames except IDR.

Definition at line 1691 of file v4l2_nv_extensions.h.

◆ v4l2_videodec_input_error_type

enum v4l2_videodec_input_error_type

Enum v4l2_videodec_input_error_type, possible error types for input stream.

Enumerator
V4L2_DEC_ERROR_NONE	no error.
V4L2_DEC_ERROR_SPS	sps error.
V4L2_DEC_ERROR_PPS	pps error.
V4L2_DEC_ERROR_SLICE_HDR	slice header error.
V4L2_DEC_ERROR_MISSING_REF_FRAME	missing reference frame error.
V4L2_DEC_ERROR_VPS	VPS error.

Definition at line 1713 of file v4l2_nv_extensions.h.

Jetson Linux API Reference

35.6.0 Release

Detailed Description

Supported Pixel Formats

Supported Memory Types

Supported Controls

Supported Events

Handling Resolution Change Events

EOS Handling

Decoder Input Frame Metadata

Decoder Output Frame Metadata

Data Structures

Macros

Typedefs

Enumerations

Macro Definition Documentation

◆ V4L2_CID_MPEG_VIDEO_DISABLE_COMPLETE_FRAME_INPUT

◆ V4L2_CID_MPEG_VIDEO_DISABLE_DPB

◆ V4L2_CID_MPEG_VIDEO_ERROR_REPORTING

◆ V4L2_CID_MPEG_VIDEO_SKIP_FRAMES

◆ V4L2_CID_MPEG_VIDEODEC_INPUT_METADATA

◆ V4L2_CID_MPEG_VIDEODEC_METADATA

◆ V4L2_CID_MPEG_VIDEODEC_SAR_HEIGHT

◆ V4L2_CID_MPEG_VIDEODEC_SAR_WIDTH

◆ V4L2_CID_MPEG_VIDEOENC_DS_SEI_DATA

◆ V4L2_CID_VIDEODEC_DISPLAYDATA_PRESENT

◆ V4L2_CID_VIDEODEC_HDR_MASTERING_DISPLAY_DATA

Typedef Documentation

◆ v4l2_ctrl_h264dec_bufmetadata

◆ v4l2_ctrl_hevcdec_bufmetadata

◆ v4l2_ctrl_videodec_currentframe_metadata

◆ v4l2_ctrl_videodec_dpbinfometadata

◆ v4l2_ctrl_videodec_inputbuf_metadata

◆ v4l2_ctrl_videodec_outputbuf_metadata

◆ v4l2_ctrl_videodec_refframe_metadata

◆ v4l2_ctrl_videodec_statusmetadata

Enumeration Type Documentation

◆ v4l2_cuda_mem_type

◆ v4l2_skip_frames_type

◆ v4l2_videodec_input_error_type