Gstreamer Plugin and Application Migration

The buffer data type used in DeepStream, NvBufSurface, has changed significantly from DeepStream 3.0 to DeepStream 4.0. The interface now abstracts platform-specific details of memory for the dGPU and Jetson platforms. It also defines functions for allocating and freeing memory, and for mapping and unmapping buffers in the CPU and GPU address spaces.

The following subsections summarize important differences between memory-related operations in the DeepStrem 3.0 and 4.0 APIs.

Memory Allocation

DeepStream 3.0 (dGPU): Call cudaMalloc().

DeepStream 3.0 (Jetson): Call NvBufferCreate().

DeepStream 4.0: Call NvBufSurfaceCreate().

Memory Release

DeepStream 3.0 (dGPU): Call cudaFree().

DeepStream 3.0 (Jetson): Call NvBufferDestroy().

DeepStream 4.0: Call NvBufSurfaceDestroy().

Resolution

DeepStream 3.0 (dGPU): Specified by NvBufSurface.width and NvBufSurface.height.

DeepStream 3.0 (Jetson): Create and fill an NvBufferParams structure:

NvBufferParams params;

NvBufferGetParams(NvBufSurface.buf_fds[i], &params);

After the call, params.width and params.height specify the resolution.

DeepStream 4.0: Specified by NvBufSurface.width and NvBufSurface.height.

Color Format

DeepStream 3.0 (dGPU): Passed as GStreamer capabilities.

DeepStream 3.0 (Jetson): After a call to NvBufferGetParams() (see Resolution), specified by params.pixel_format.

DeepStream 4.0: Specified by NvBufSurface.surfaceList[i].colorFormat.

Per-Plane Parameters

DeepStream 3.0 (dGPU): Inferred from color format information and resolution, e.g. for NV12:

widthUV = NvBufSurface.width / 2

DeepStream 3.0 (Jetson): After a call to NvBufferGetParams() (see Resolution), specified by params.width[planeIndex], params.height[planeIndex], and params.pitch[planeIndex].

DeepStream 4.0: Specified by NvBufSurface.surfaceList[i].planeParams.

Pitch

DeepStream 3.0 (dGPU): Inferred from color format and width, e.g. for RGBA:

pitch = width * 4

DeepStream 3.0 (Jetson): After a call to NvBufferGetParams() (see Resolution), specified by params.pitch[planeIndex].

DeepStream 4.0: Specified by NvBufSurface.surfaceList[i].planeParams.pitch[planeIndex].

Mapping and Unmapping in CPU Address Space

DeepStream 3.0 (dGPU): Specified by NvBufSurface.buf_data[i].

Applies only to managed (unified) and host allocated memories.

DeepStream 3.0 (Jetson): After a call to NvBufferGetParams() (see Resolution), to map a plane of i'th buffer in batch:

NvBufferMemMap(NvBufSurface.buf_fds[i], NvBufferMem_Read_Write,

planeIndex, &cpuAddr);

The corresponding address in CPU address space is in cpuAddr.

To unmap the memory:

NvBufferMemUnmap(NvBufSurface.buf_fds[i], planeIndex, &cpuAddr);

DeepStream 4.0: To map a plane of i'th buffer in batch:

NvBufSurfaceMap(NvBufSurface, i, planeIndex, NVBUF_MAP_READ_WRITE);

The corresponding address in CPU address space is NvBufSurface.surfaceList[i].mappedAddr.addr[planeIndex].

To unmap the memory:

NvBufSurfaceUnmap(NvBufSurface, i, planeIndex);

Note:

CUDA device allocated memory cannot be accessed in the CPU address space.

Mapping and Unmapping Memory in GPU Address Space

DeepStream 3.0 (dGPU): Specified by NvBufSurface.buf_data[i].

DeepStream 3.0 (Jetson): After a call to NvBufferGetParams() (see Resolution), to map i'th buffer in batch:

egl_images[i] = NvEGLImageFromFd(eglDisplay, vBufSurface.buf_fds[i])

cuGraphicsEGLRegisterImage(&cuda_resources[i],egl_images[i],

CU_GRAPHICS_MAP_RESOURCE_FLAGS_NONE);

cuGraphicsResourceGetMappedEglFrame(&egl_frames[i],

cuda_resources[i], 0, 0);

The address in GPU (CUDA) address space is:

egl_frames[i].frame.pPitch[planeIndex];

To unmap the memory:

cuGraphicsUnregisterResource(cuda_resources[i]);

NvDestroyEglImage(eglDisplay, eglImage);

DeepStream 4.0 (dGPU): The address of the i'th buffer in batch is specified by NvBufSurface.surfaceList[i].dataPtr.

DeepStream 4.0 (Jetson): To map the address of the i'th buffer in batch:

NvBufSurfaceMapEglImage(NvBufSurface, i);

cuGraphicsEGLRegisterImage(&cuda_resources[i],

NvBufSurface.surfaceList[i].mappedAddr.eglImage,

CU_GRAPHICS_MAP_RESOURCE_FLAGS_NONE);

cuGraphicsResourceGetMappedEglFrame(&egl_frames[i],

cuda_resources[i], 0, 0);

Unified Image/Video Transformation: NvBufSurfTransform

DeepStream 3.0 used platform-specific APIs for transformations: on dGPU, NPP/custom CUDA kernels; on Jetson, NvBufferTransform() from the nvbuf_utils library.

In DeepStream 4.0, the NvBufSurfTransform functions provide a unified interface for video/image transformations. These functions are platform-agnostic (except for configuring the API), reducing the need for platform-specific coding in applications.

Metadata Changes

For more information about metadata changes, see the chapter “Metadata Migration from DeepStream Release 3.0 to 4.0” in DeepStream 4.0 Plugin Manual.

Gst Element Factory Name Changes

The following element factory names are changed from DeepStream 3.0>

• nvdec_h264 changed to nvv4l2decoder

• nvvidconv changed to nvvideoconvert

• nvosd changed to nvdsosd

For more details about the API and examples of use, see the Gst-DsExample code (sources/gst-plugins/gst-dsexample).

Also refer to the Gst-nvinfer source code (sources/gst-plugins/gst-nvinfer), and the descriptions of NvBufSurface, NvBufSurfTransform(), and Metadata in DeepStream SDK API Reference.