MultimediaExtension

Extension for multimedia related data types, interfaces and components in GXF Core.

  • UUID: 6f2d1afc-1057-481a-9da6-a5f61fed178e

  • Version: 2.4.0

  • Author: NVIDIA

  • License: LICENSE

Components

nvidia::gxf::AudioBuffer

AudioBuffer is similar to Tensor component in the standard extension and holds memory and metadata corresponding to an audio buffer.

  • Component ID: a914cac6-5f19-449d-9ade-8c5cdcebe7c3

AudioBufferInfo structure captures the following metadata:

Field

Description

channels

Number of channels in an audio frame

samples

Number of samples in an audio frame

sampling_rate

sampling rate in Hz

bytes_per_sample

Number of bytes required per sample

audio_format

AudioFormat of an audio frame

audio_layout

AudioLayout of an audio frame

Supported AudioFormat types:

AudioFormat

Description

GXF_AUDIO_FORMAT_S16LE

16-bit signed PCM audio

GXF_AUDIO_FORMAT_F32LE

32-bit floating-point audio

Supported AudioLayout types:

AudioLayout

Description

GXF_AUDIO_LAYOUT_INTERLEAVED

Data from all the channels to be interleaved - LRLRLR

GXF_AUDIO_LAYOUT_NON_INTERLEAVED

Data from all the channels not to be interleaved - LLLRRR

nvidia::gxf::VideoBuffer

VideoBuffer is similar to Tensor component in the standard extension and holds memory and metadata corresponding to a video buffer.

  • Component ID: 16ad58c8-b463-422c-b097-61a9acc5050e

VideoBufferInfo structure captures the following metadata:

Field

Description

width

width of a video frame

height

height of a video frame

color_format

VideoFormat of a video frame

color_planes

ColorPlane(s) associated with the VideoFormat

surface_layout

SurfaceLayout of the video frame

Supported VideoFormat types:

VideoFormat

Description

GXF_VIDEO_FORMAT_YUV420

BT.601 multi planar 4:2:0 YUV

GXF_VIDEO_FORMAT_YUV420_ER

BT.601 multi planar 4:2:0 YUV ER

GXF_VIDEO_FORMAT_YUV420_709

BT.709 multi planar 4:2:0 YUV

GXF_VIDEO_FORMAT_YUV420_709_ER

BT.709 multi planar 4:2:0 YUV ER

GXF_VIDEO_FORMAT_NV12

BT.601 multi planar 4:2:0 YUV with interleaved UV

GXF_VIDEO_FORMAT_NV12_ER

BT.601 multi planar 4:2:0 YUV ER with interleaved UV

GXF_VIDEO_FORMAT_NV12_709

BT.709 multi planar 4:2:0 YUV with interleaved UV

GXF_VIDEO_FORMAT_NV12_709_ER

BT.709 multi planar 4:2:0 YUV ER with interleaved UV

GXF_VIDEO_FORMAT_RGBA

RGBA-8-8-8-8 single plane

GXF_VIDEO_FORMAT_BGRA

BGRA-8-8-8-8 single plane

GXF_VIDEO_FORMAT_ARGB

ARGB-8-8-8-8 single plane

GXF_VIDEO_FORMAT_ABGR

ABGR-8-8-8-8 single plane

GXF_VIDEO_FORMAT_RGBX

RGBX-8-8-8-8 single plane

GXF_VIDEO_FORMAT_BGRX

BGRX-8-8-8-8 single plane

GXF_VIDEO_FORMAT_XRGB

XRGB-8-8-8-8 single plane

GXF_VIDEO_FORMAT_XBGR

XBGR-8-8-8-8 single plane

GXF_VIDEO_FORMAT_RGB

RGB-8-8-8 single plane

GXF_VIDEO_FORMAT_BGR

BGR-8-8-8 single plane

GXF_VIDEO_FORMAT_R8_G8_B8

RGB - unsigned 8 bit multiplanar

GXF_VIDEO_FORMAT_B8_G8_R8

BGR - unsigned 8 bit multiplanar

GXF_VIDEO_FORMAT_GRAY

8 bit GRAY scale single plane

GXF_VIDEO_FORMAT_GRAY16

16 bit GRAY scale single plane

GXF_VIDEO_FORMAT_GRAY32

32 bit GRAY scale single plane

GXF_VIDEO_FORMAT_GRAY32F

float 32 bit GRAY scale single plane

GXF_VIDEO_FORMAT_RGB16

RGB-16-16-16 single plane

GXF_VIDEO_FORMAT_BGR16

BGR-16-16-16 single plane

GXF_VIDEO_FORMAT_RGB32

RGB-32-32-32 single plane

GXF_VIDEO_FORMAT_BGR32

BGR-32-32-32 single plane

GXF_VIDEO_FORMAT_R16_G16_B16

RGB - signed 16 bit multiplanar

GXF_VIDEO_FORMAT_B16_G16_R16

BGR - signed 16 bit multiplanar

GXF_VIDEO_FORMAT_R32_G32_B32

RGB - signed 32 bit multiplanar

GXF_VIDEO_FORMAT_B32_G32_R32 | BGR - signed 32 bit multiplanar

GXF_VIDEO_FORMAT_NV24 | multi planar 4:4:4 YUV with interleaved UV

GXF_VIDEO_FORMAT_NV24_ER

multi planar 4:4:4 YUV ER with interleaved UV

GXF_VIDEO_FORMAT_R8_G8_B8_D8

RGBD unsigned 8 bit multiplanar

GXF_VIDEO_FORMAT_R16_G16_B16_D16

RGBD unsigned 16 bit multiplanar

GXF_VIDEO_FORMAT_R32_G32_B32_D32

RGBD unsigned 32 bit multiplanar

GXF_VIDEO_FORMAT_RGBD8

RGBD 8 bit unsigned single plane

GXF_VIDEO_FORMAT_RGBD16

RGBD 16 bit unsigned single plane

GXF_VIDEO_FORMAT_RGBD32

RGBD 32 bit unsigned single plane

GXF_VIDEO_FORMAT_D32F

Depth 32 bit float single plane

GXF_VIDEO_FORMAT_D64F

Depth 64 bit float single plane

GXF_VIDEO_FORMAT_RAW16_RGGB

RGGB-16-16-16-16 single plane

GXF_VIDEO_FORMAT_RAW16_BGGR

BGGR-16-16-16-16 single plane

GXF_VIDEO_FORMAT_RAW16_GRBG

GRBG-16-16-16-16 single plane

GXF_VIDEO_FORMAT_RAW16_GBRG

GBRG-16-16-16-16 single plane

Supported SurfaceLayout types:

SurfaceLayout

Description

GXF_SURFACE_LAYOUT_PITCH_LINEAR

pitch linear surface memory

GXF_SURFACE_LAYOUT_BLOCK_LINEAR

block linear surface memory