Gst-nvvideo4linux2

DeepStream extends the open source V4L2 codec plugins (here called Gst-v4l2) to support hardware-accelerated codecs.

Gst-nvvideo4linux2

Decoder

The OSS Gst-nvvideo4linux2 plugin leverages the hardware decoding engines on Jetson and DGPU platforms by interfacing with libv4l2 plugins on those platforms. It supports H.264, H.265, AV1, JPEG and MJPEG formats. The plugin accepts an encoded bitstream and uses the NVDEC hardware engine to decode the bitstream. The decoded output can be NV12 or YUV444 format which depends on the encoded stream content.

Note

When you use the v4l2 decoder for decoding JPEG images, you must use the open source jpegparse plugin before the decoder to parse encoded JPEG images.

Inputs and Outputs

  • Inputs

    • An encoded bitstream. Supported formats are H.264, H.265, AV1, JPEG and MJPEG

    • H264 encoded bitstream in 8bit 4:2:0 format.

    • H265 encoded bitstream in 8/10/12 bit 4:2:0 and 8/10/12 bit 4:4:4 format.

    • AV1 encoded bitstream in 8bit 4:2:0 format.

  • Output

    • H264 decoder outputs GstBuffer in 8bit semi-planar(NV12) 4:2:0 format.

    • H265 decoder outputs GstBuffer in 8/10/12 bit semi-planar(NV12) 4:2:0 format, 8/10/12 bit planar(YUV444) 4:4:4 format.

    • AV1 decoder outputs GstBuffer in 8bit semi-planar(NV12) 4:2:0 format.

Features

The following table summarizes the features of the Gst-nvvideo4linux2 decoder plugin.

Gst-nvvideo4linux2 plugin decoder features

Feature

Description

Supports H.264 decode

h264 encoded bitstream is decoded into 8bit NV12

Supports H.265 decode

h265 encoded bitstream is decoded into 8/10/12 bit NV12 or 8/10/12 bit YUV444 depending on the stream content

Supports JPEG/MJPEG decode

User-configurable CUDA memory type (Pinned/Device/Unified) for output buffers

Note

  • 10/12bit YUV444 format support is applicable only for x86/dGPU platforms.

  • Also, for 12bit NV12 format, the output caps shows I420_12LE. But the data is handled as semiplanar 12 bit data. This is a workaround since gstreamer v1.16 does not have the particular caps for 12 bit NV12. Downstream component should process it considering as 12bit NV12 only.

Gst Properties

The following table summarizes the Gst properties of the Gst-nvvideo4linux2 decoder plugin.

Gst-nvvideo4linux2 plugin decoder gst properties

Property

Meaning

Type and Range

Example/Notes

Platforms

capture-io-mode

Capture I/O mode (matches src pad)

(0): auto - GST_V4L2_IO_AUTO

(2): mmap - GST_V4L2_IO_MMAP

On dGPU:

Enum “GstNvV4l2IOMode”

On Jetson:

Enum “GstNvV4l2DecCaptureIOMode”

Default: 0

dGPU Jetson

capture-buffer-dynamic-allocation

Set to enable capture buffer dynamic allocation

(0): cap_buf_dyn_alloc_disabled - Capture buffer dynamic allocation disabled

(1): fw_cap_buf_dyn_alloc_enabled - Capture buffer dynamic allocation enabled for forward playback

(2): rw_cap_buf_dyn_alloc_enabled - Capture buffer dynamic allocation enabled for reverse playback

(3): fw_rw_cap_buf_dyn_alloc_enabled - Capture buffer dynamic allocation enabled for forward and reverse playback

Enum “CaptureBufferDynamicAllocationModes”

Default: 0

Jetson

cudadec-memtype

Memory type for CUDA decoder buffers.

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

Enum “CudaDecMemType”

Default: 2

dGPU

disable-dpb

Set to disable DPB buffer for low latency

Boolean

Default: false

Jetson

drop-frame-interval

Interval to drop the frames, e.g. a value of 5 means the decoder outputs every fifth frame, and others are dropped.

Integer, 1 to 30

Default: 0

dGPU Jetson

extract-sei-type5-data

Set to extract and attach SEI type5 unregistered data on output buffer

Boolean

Default: false

dGPU

enable-error-check

Set to enable error check

Boolean

Default: false

Jetson

enable-frame-type-reporting

Set to enable frame type reporting

Boolean

Default: false

Jetson

enable-full-frame

Whether or not the data is full framed

Boolean

Default: false

Jetson

enable-max-performance

Set to enable max performance

Boolean

Default: false

Jetson

gpu-id

Device ID of GPU to use for decoding.

Integer

Default:0

dGPU

low-latency-mode

Set low latency mode for bitstreams having I and IPPP frames with no B frames

Boolean

Default: false

dGPU

mjpeg

Set to open MJPEG block

Boolean

Default: false

Jetson

num-extra-surfaces

Number of surfaces in addition to min decode surfaces given by the V4L2 driver.

Integer, 1 to 24

Default: 0

dGPU Jetson

output-io-mode

Output side I/O mode (matches sink pad)

On dGPU:

(0): auto - GST_V4L2_IO_AUTO

(2): mmap - GST_V4L2_IO_MMAP

(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT

On Jetson:

(0): auto - GST_V4L2_IO_AUTO

(2): mmap - GST_V4L2_IO_MMAP

(3): userptr - GST_V4L2_IO_USERPTR

On dGPU:

Enum “GstNvV4l2IOMode”

On Jetson:

Enum “GstNvV4l2DecOutputIOMode”

Default: 0, “auto”

dGPU Jetson

sei-uuid

Set 16 bytes UUID string for SEI Parsing, extract-sei-type5-data should be TRUE

String

Default: null

dGPU

skip-frames

Type of frames to skip during decoding.

0 (decode_all): decode all frames

1 (decode_non_ref): skips non-ref frames (Applicable only on Jetson platform)

2 (decode_key): decode key frames

Enum “SkipFrame”

Default: 0

dGPU Jetson

Encoder

The OSS Gst-nvvideo4linux2 plugin leverages the hardware accelerated encoding engine available on Jetson and dGPU platforms by interfacing with libv4l2 plugins on those platforms. The plugin accepts RAW data in I420/NV12 or YUV444 format. It uses the NVENC hardware engine to encode RAW input. Encoded output is in elementary bitstream supported format. To set appropriate tuning-info-id and preset-id properties please refer to Nvidia Video Codec SDK Documentation at https://docs.nvidia.com/video-technologies/video-codec-sdk/11.1/nvenc-preset-migration-guide/index.html

Inputs and Outputs

  • Inputs

    • RAW input in I420/NV12 or YUV444 format

    • 8 bit I420/NV12 and YUV444 raw formats

    • 8/10 bit I420/NV12 and YUV444 raw formats

  • Output

    • Gst Buffer with encoded output in H264, H265 and AV1 format.

Features

The following table summarizes the features of the Gst-nvvideo4linux2 encoder plugin.

Gst-nvvideo4linux2 plugin encoder features

Feature

Description

Supports H.264 encode

8 bit I420/NV12, 8bit YUV444 raw format can be encoded into h264 bitstream

Supports H.265 encode

8/10 bit I420/NV12, 8/10 bit YUV444 raw format can be encoded into h265 bitstream

Note

  • 8/10bit YUV444 format support is applicable only for x86/dGPU platforms.

Gst Properties

The following table summarizes the Gst properties of the Gst-nvvideo4linux2 encoder plugin.

Gst-nvvideo4linux2 plugin encoder gst properties

Property

Meaning

Type and Range

Example/Notes

Platforms

aq

Enable spatial AQ and set its stength

Unsigned Integer, 0-15

Default: 0, “auto”

dGPU

bit-packetization

Whether or not Packet size is based upon Number Of bits

Boolean

Default: false

Jetson

bitrate

Sets bitrate for encoding, in bits/seconds.

Unsigned Integer

Default: 4000000

dGPU Jetson

capture-io-mode

Capture I/O mode (matches src pad)

(0): auto - GST_V4L2_IO_AUTO

(2): mmap - GST_V4L2_IO_MMAP

(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT

Enum “GstNvV4l2IOMode”

Default: 0, “auto”

dGPU Jetson

constqp

Sets constant QP values for I:P:B frames

String, 1 to 51

Default: null

dGPU

copy-meta

Copies input metadata on output buffer

Boolean

Default: false

dGPU

copy-timestamp

Enabling the flag allows to copy the timestamps from input to the capture stream.

Boolean

Default: false

Jetson

control-rate

Sets rate control mode for v4l2 encoding

(0): variable_bitrate - GST_V4L2_VIDENC_VARIABLE_BITRATE

(1): constant_bitrate - GST_V4L2_VIDENC_CONSTANT_BITRATE

(2): constantQP - GST_V4L2_VIDENC_CONSTANT_QP

Enum “GstV4l2VideoEncRateControlType”

Default: 1, “constant_bitrate”

dGPU Jetson

cq

Sets targetQuality (Constant Quality) level for VBR mode

Integer, 0-51

Default: 0, “auto”

dGPU

disable-cabac

Set Entropy Coding Type CAVLC(TRUE) or CABAC(FALSE)

Boolean

Default: false

Jetson

enable-lossless

Enable lossless encoding for YUV444

Boolean

Default: false

Jetson

EnableMVBufferMeta

Enable Motion Vector Meta data for encoding

Boolean

Default: false

Jetson

EnableTwopassCBR

Enable two pass CBR while encoding

Boolean

Default: false

Jetson

extended-colorformat

Sets extended colorformat pixel values 0 to 255 in VUI Info

Boolean

Default: false

dGPU

force-idr

Force an IDR frame

Boolean

Default: false

dGPU

force-intra

Forces an INTRA frame

Boolean

Default: false

dGPU

gpu-id

Device ID of GPU to used.

Unsigned Integer

Default:0

dGPU

idrinterval

Sets IDR Frame occurence frequency in encoded stream

Unsigned Integer

Default: 256

dGPU Jetson

iframeinterval

Sets encoding intra-frame occurrence frequency.

Unsigned Integer

Default: 30

dGPU Jetson

initqp

Set values of initQP in I:P:B order. This provides rough hint to encoder to influence the qp difference between I, P and B

String

Default: null

dGPU

insert-aud

Insert H.264 Access Unit Delimiter(AUD)

Boolean

Default: false

Jetson

insert-sps-pps

Insert H.264 SPS, PPS at every IDR frame

Boolean

Default: false

Jetson

insert-vui

Insert H.264 VUI(Video Usability Information) in SPS

Boolean

Default: false

Jetson

intra-refresh

Intra Refresh Parameters. Use string with unsigned integer values of Intra Refresh in enableIntraRefresh,intraRefreshPeriod,intraRefreshCnt in same order to set the property. e.g. 1,30,5

String

Default: null

dGPU

maxbitrate

Set maxbitrate for v4l2 encoder. This mode is specified by setting rateControlMode to VBR

Unsigned Integer

Default: 0

dGPU

maxperf-enable

Enable or Disable Max Performance mode

Boolean

Default: false

Jetson

MeasureEncoderLatency

Enable Measure Encoder latency Per Frame

Boolean

Default: false

Jetson

num-B-Frames

Number of B Frames between two reference frames (not recommended)

Unsigned Integer, 0-2

Default: 0

Jetson

num-Ref-Frames

Number of Reference Frames for encoder

Unsigned Integer, 0-8

Default: 1

Jetson

output-io-mode

Sets output side I/O mode (matches sink pad)

(0): auto - GST_V4L2_IO_AUTO

(2): mmap - GST_V4L2_IO_MMAP

(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT

Enum “GstNvV4l2IOMode”

Default: 0, “auto”

dGPU Jetson

peak-bitrate

Peak bitrate in variable control-rate. The value must be >= bitrate (1.2*bitrate) is set by default (Default: 0)

Unsigned Integer

Default: 0

Jetson

poc-type

Set Picture Order Count type value

Unsigned Integer, 0-2

Default: 1

Jetson

preset-id

Sets CUVID Preset ID for Encoder

Unsigned Integer, 1 to 7

Default: 1

dGPU

preset-level

HW preset level for encoder.

(0): DisablePreset - Disable HW-Preset

(1): UltraFastPreset - UltraFastPreset for high perf

(2): FastPreset - FastPreset

(3): MediumPreset - MediumPreset

(4): SlowPreset - SlowPreset

Enum “GstV4L2VideoEncHwPreset”

Default: 1, “UltraFastPreset”

Jetson

Profile

H.264/H.265 encoder profile.

For H.264:

(0): Baseline - GST_V4L2_H264_VIDENC_BASELINE_PROFILE

(2): Main - GST_V4L2_H264_VIDENC_MAIN_PROFILE

(4): High - GST_V4L2_H264_VIDENC_HIGH_PROFILE

(7): High444 - GST_V4L2_H264_VIDENC_HIGH_444_PREDICTIVE

For H.265:

(0): Main - GST_V4L2_H265_VIDENC_MAIN_PROFILE

(1): Main10 - GST_V4L2_H265_VIDENC_MAIN10_PROFILE

(3): FREXT - GST_V4L2_H265_VIDENC_FREXT_PROFILE

Enum “GstV4l2VideoEncProfileType”

For H.264:

Default: 0 (Baseline)

For H.265:

Default: 0 (Main)

dGPU Jetson

Property

Meaning

Type and Range

Example/Notes

Platforms

qp-range

Sets quantization paramenter range for P, I and B frame

String, 1 to 51

Default: null

dGPU Jetson

quant-b-frames

Quantization parameter for B-frames (0xffffffff=component default), use with ratecontrol-enable = 0 and preset-level = 0

Unsigned Integer, 0 - 4294967295

Default: 4294967295

Jetson

quant-i-frames

Quantization parameter for I-frames (0xffffffff=component default), use with ratecontrol-enable = 0

Unsigned Integer, 0 - 4294967295

Default: 4294967295

Jetson

quant-p-frames

Quantization parameter for P-frames (0xffffffff=component default), use with ratecontrol-enable = 0 and preset-level = 0

Unsigned Integer, 0 - 4294967295

Default: 4294967295

Jetson

ratecontrol-enable

Enable or Disable rate control mode

Boolean

Default: true

Jetson

slice-header-spacing

Slice Header Spacing number of macroblocks/bits in one packet

Unsigned Integer64

Default: 0

Jetson

SliceIntraRefreshInterval

Set SliceIntraRefreshInterval

Unsigned Integer

Default: 0

Jetson

temporalaq

Set this to enable adaptive quantization (Temporal).

Boolean

Default: false

dGPU

tuning-info-id

Sets Tuning Info Preset for encoder.

(1): HighQualityPreset - Tuning Preset for High Quality

(2): LowLatencyPreset - Tuning Preset for Low Latency

(3): UltraLowLatencyPreset - Tuning Preset for Low Latency

(4): LosslessPreset - Tuning Preset for Lossless

Enum “GstV4L2VideoEncTuingInfoPreset”

Default: 2 (LowLatencyPreset)

dGPU

vbvbufsize

Set VBV buffer size in bits.

Unsigned Integer

Default: 0 (default value by encoder)

dGPU

vbvinit

Set VBV initial delay in bits

Unsigned Integer

Default: 0 (default value by encoder)

dGPU

vbv-size

virtual buffer size

Unsigned Integer

Default: 4000000

Jetson