Gst-nvdsucx

Gst-NvDsUcx is a Gstreamer plugin that provides a set of elements that can be used to send and receive pipeline data using RDMA. This allows for distributing the Gstreamer pipeline to various hosts in order to use distributed GPU resources. It is built on top of the Unified Communication X (UCX) library to send/receive Gstreamer packets over an RDMA-enabled network. UCX is an open-source library that accelerates data over high-performance networks and can utilize GPUDirect RDMA technology for minimal network latencies and highest throughput of distributed GPU traffic. For more details on UCX, see https://openucx.org.

Description

Gst-NvDsUcx provides separate sink (to receive data from the pipeline) and source elements (to forward data to the pipeline), which connect to each other over the RDMA network. Furthermore, each sink or source type element can be a server or client, where the server element must be started before the client. As a result, the Gst-NvDsUcx plugin provides 4 elements: nvdsucxserversink, nvdsucxclientsink, nvdsucxserversrc, nvdsucxclientsrc.

Since the Gst-NvDsUcx plugin needs to present itself as a sink and source to the Deepstream pipeline, you need to pair the elements based on which part of the pipeline needs to be started first:

  • nvdsucxserversink <-> nvdsucxclientsrc (Sink side starts first)

  • nvdsucxclientsink <-> nvdsucxserversrc (Source side starts first)


Gst-nvdsucx

Requirements

The Gst-NvDsUcx plugin has the following requirements (in addition to the Deepstream 6.3 SDK requirements):

  • NVIDIA ConnectX6-DX NIC or later.

  • Mellanox Open Fabrics Enterprise Distribution (MLNX_OFED) - version 5.5 or later, see https://network.nvidia.com/products/infiniband-drivers/linux/mlnx_ofed/

    • For installation instructions, see https://docs.nvidia.com/networking/display/MLNXOFEDv551032/Installing+MLNX_OFED

    • If installing the Mellanox OFED within a container:

      • Make sure to install the kernel drivers in the host OS by passing the --all flag to mlnxofedinstall script.

      • In the container you can only install the user space libraries using the --user-space-only flag to the mlnxofedinstall script.

  • UCX - version 1.13 or later - needs to be compiled with CUDA support or use CUDA-enabled UCX packages from the git repository directly, see https://github.com/openucx/ucx/releases

    • For installation instructions, follow the Release build instructions from here: https://github.com/openucx/ucx#release-builds. Note that UCX library should be compiled with CUDA as follows:

      $ ./contrib/configure-release --prefix=/install/path --enable-examples --with-java=no --with-cuda=/path/to/cuda --enable-mt
      
  • Docker container support

    • If you wish to use the plugin inside a container, make sure to add the following flags during docker run command:

      • --privileged --network host

      • --cap-add CAP_SYS_PTRACE --shm-size="8g"

      • --device=/dev/infiniband/uverbs0

      • --device=/dev/infiniband/rdma_cm

      • --ipc=host

      • -e CUDA_CACHE_DISABLE=0

      • -v /dev/infiniband:/dev/infiniband

  • For additional metadata processing, Gst-NvDsUcx depends on the serialization library provided by the Gst-NvDsMetaUtils plugin.Refer to the Gst-NvDsMetaUtils documentation for configuring and installing the serialization library.


Note

This plugin is only supported on x86_64 platforms.

Inputs and Outputs

  • Inputs (for Nvdsucxserversink or Nvdsucxclientsink)

    Any one of the following:

    • NV12/RGBA NVMM Gst Buffer + (NvDsBatchMeta + Serialized NvDsUserMeta/Gst Meta - optional)

    • NVMM or Raw Audio Buffers + (NvDsBatchMeta - optional)

    • Raw Text Gst Buffers

  • Control parameters

    • addr

    • port

    • buf-type

    • gpu-id

    • raw-buf-size

    • nvbuf-memory-type

    • num-nvbuf

    • nvbuf-batch-size

    • num-conns

  • Output (from Nvdsucxserversrc or Nvdsucxclientsrc)

    Any one of the following:

    • NV12/RGBA NVMM Gst Buffer + (NvDsBatchMeta + Serialized Video NvDsUserMeta/Gst Meta - optional)

    • NVMM or Raw Audio Buffers + (NvDsBatchMeta + Serialized Audio NvDsUserMeta/Gst Meta - optional)

    • Raw Text Gst Buffers

Gst Properties

The Gst-nvdsucx plugin has the following properties based on which type of element is used:

Gst-nvdsucx gst properties

Property

Type of Element

Description

Type and Range

Examples

addr

Server

The IP address to which a client will connect

String

Default: 127.0.0.1

addr = 192.168.100.1

addr

Client

The server IP address

String

Default: 127.0.0.1

addr = 192.168.100.1

port

Server

Listening port for connections from clients

Integer 0 - 66535

Default: 7174

port = 4000

port

Client

The server port number

Integer 0 - 65535

Default: 7174

port = 4000

buf-type

All

Type of data handled by UCX:

0 - video

1 - audio

2 - raw-audio

4 - text

Default: 0

Integer

buf-type = 0

gpu-id

Source

GPU ID to use

Integer 0 - 4294967295

Default: 0

gpu-id=0

raw-buf-size

All

Size of raw buffer to allocate

Integer 0 - 8192

Default: 8192

raw-buf-size=1024

nvbuf-memory-type

Source

Type of NvBufSurface Memory to allocate for output buffers

0 - Default memory

1 - cuda-pinned (Allocate Pinned/Host Cuda Memory)

2 - cuda-device (Allocate Device cuda Memory)

3 - cuda-unified (Allocate unified cuda memory)

Default: 3

Integer

nvbuf-memory-type = 2

num-nvbuf

Source

The number of Nv Buffers to allocate

Integer 0 - 10

Default: 4

num-nvbuf = 8

nvbuf-batch-size

All

The maximal batch size of a Nv Buffer

Integer 1 - 2147483647

Default: 1

nvbuf-batch-size = 4

num-conns

ServerSink

The number of client connections to expect 1

Integer 1 - 4

Default: 1

num-conns = 2

Footnote

1

These connections are established synchronously. The serversink plugin will always wait till all clients connect before starting the pipeline. Only, the serversink plugin supports more than one clientsrc connecting. The serversrc plugin will support only 1 connection from a clientsink.

Examples

The DeepStream SDK 6.1+ includes three examples on how to use the Gst-NvDsUcx plugin to disaggregate/divide the Gstreamer pipeline to run on separate processes/servers. Note that each example has a server and client program to run different parts of the pipeline separately. Always start the server program before the client program.

Example 1:

The example here shows how to send/receive video data in the Gstreamer pipeline using serversink and clientsrc elements of the Gst-NvDsUcx plugin. The pipeline uses the uridecodebin and the nvvideoconverter plugins to pass the video frames to the serversink element based on the caps filter. The serversink forwards this video data to the clientsrc element (on another node/process using RDMA), which then forwards the data to the video converter. Finally, the data is stored in a file after encoding.

On DS Node 1:

gst-launch-1.0 uridecodebin uri="file:///sample_1080p.mp4" async-handling=1 name=src1 src1. ! \
queue ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080' ! \
nvdsucxserversink addr=192.168.100.1 port=4000 buf-type=nvdsucx-buf-video

On DS Node 2:

gst-launch-1.0 nvdsucxclientsrc addr=192.168.100.1 port=4000 nvbuf-memory-type=2 num-nvbuf=4 buf-type=nvdsucx-buf-video ! \
'video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=30/1' ! \
queue ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! qtmux name=mux_0 ! \
filesink sync=1 async=0 qos=0 location=~/out_1080p.mp4

Example 2:

This example shows how to distribute the DS pipeline using Gst-NvDsUcx plugin and use serialization / de-serialization components to send serialized data over RDMA network. The Deepstream pipeline here consists of the streammux plugin that takes input from the filesrc after decoding. The streammux passes the frames to the nvinfer plugin that identifies certain objects in the frames and add that metadata to the frame. The serialization plugin (part of the Gst-NvDsMetaUtils library) creates a binary object corresponding to the metadata and adds it to the frame. The clientsink and serversrc elements are used here to demonstrate the flexiblity of setup of the Gst-NvDsUcx here. The clientsink will send the additional metadata along with the video frame via RDMA to the serversrc.

The serversrc then forwards the data to the deserialization plugin which extracts it to append the metadata correctly to the frame. The nvdsosd plugin interprets the metadata (bounding boxes) and then the file is stored after encoding.

On DS Node 1:

gst-launch-1.0 filesrc location=~/sample_1080p.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 ! \
nvvideoconvert ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-6.1/samples/configs/deepstream-app/config_infer_primary.txt ! \
nvdsmetainsert serialize-lib = "/opt/nvidia/deepstream/deepstream-6.1/lib/libnvds_video_metadata_serialization.so" ! \
nvdsucxclientsink addr=192.168.100.1 port=4000 buf-type=nvdsucx-buf-video

On DS Node 2:

gst-launch-1.0 nvdsucxserversrc addr=192.168.100.1 port=4000 nvbuf-memory-type=2 num-nvbuf=8 buf-type=nvdsucx-buf-video nvbuf-batch-size=1 ! \
'video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=30/1' ! nvvideoconvert ! \
nvdsmetaextract deserialize-lib = "/opt/nvidia/deepstream/deepstream-6.1/lib/libnvds_video_metadata_serialization.so" ! \
nvdsosd ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=~/out_1080p.mp4

Example 3:

This example demonstrates how audio data in a DS pipeline can be distributed across processes or nodes using the Gst-NvDsUcx and the Audio metadata serialization (part of Gst-NvDsMetaUtils) plugins. The streammux plugin interprets the audio data from the audio plugins and forwards it to the Gst-NvDsUcx plugin. Similar to the video metadata serialization plugin in Example 2, the audio metadata serialization plugin creates a binary object which the serversink element forwards to the clientsrc element. The audio metadata is extracted and added to the buffer for downstream plugins to interpret.

The streammux and streamdemux plugins only support audio in the new versions, so an environment variable must be set before the example is run.

On DS Node 1:

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 uridecodebin uri="file:///sample_1080p_h264.mp4" ! audioconvert ! \
audioresample ! 'audio/x-raw,format=F32LE,rate=48000,channels=1,layout=interleaved' ! audiobuffersplit ! \
a_streammux.sink_0 nvstreammux name=a_streammux batch-size=1 sync-inputs=1 max-latency=250000000 ! \
nvdsmetainsert serialize-lib="libnvds_audio_metadata_serialization.so" ! \
nvdsucxserversink addr=192.168.100.2 port=4000 sync=1 async=0 buf-type=nvdsucx-buf-nv-audio

On DS Node 2:

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 nvdsucxclientsrc addr=192.168.100.2 port=4000 nvbuf-memory-type=2 num-nvbuf=4 buf-type=nvdsucx-buf-nv-audio ! \
'audio/x-raw(memory:NVMM),format=F32LE,rate=48000,channels=1,layout=interleaved' ! \
nvdsmetaextract deserialize-lib = "libnvds_audio_metadata_serialization.so" ! nvstreamdemux name=asd asd.src_0 ! \
audioconvert ! "audio/x-raw,format=S16LE" ! wavenc ! filesink sync=0 async=1 qos=0 location=out.wav