Gst-nvmsgconv

The Gst-nvmsgconv plugin is responsible for converting metadata into message payloads based on schema. DeepStream 5.0 supports two variations of the schema, full and minimal. The Gst-nvmsgconv plugin can be configured to use either one of the schemas. By default, the plugin uses the full DeepStream schema to generate the payload in JSON format. The full schema supports elaborate semantics for object detection, analytics modules, events, location, and sensor. Each payload has information about a single object. You can use the minimal variation of the schema to communicate minimal information with the back end. This provides a small footprint for the payload to be transmitted from DeepStream to a message broker. Each payload can have information for multiple objects in the frame.

Gst-nvmsgconv plugin interface with a low level nvmsgconv library which provides APIs for payload generation using below metadata types: 1. NVDS_EVENT_MSG_META (NvDsEventMsgMeta) type metadata attached to the buffer as user metadata of frame meta. For the batched buffer, metadata of all objects of a frame must be under the corresponding frame meta. This is the default option. OR 2. By parsing the NVDS_FRAME_META (NvDsFrameMeta) type and NVDS_OBJECT_META (NvDsObjectMeta) type in Gst buffer and available fields in these metadata types are used to create message payload based on schema type. To use this option, set gst property msg2p-newapi = true. This API also supports attaching a custom user message blob(a json formatted string) for each payload if the msg blob is added as user metadata of NvDsFrameMeta. Additionally, with this API there’s also an other gst property you can use to set the frame interval at which each payload is generated. frame-interval = <interval>. If this is not set, the default frame interval at which frames are generates is every 30 frames

The generated payload (NvDsPayload) in both the above cases are attached back to the input buffer as NVDS_PAYLOAD_META type user metadata.

Additionally, Gst-nvmsgconv plugin provides optional properties: - debug-payload-dir : Absolute path of the directory to dump payloads for debugging - multiple-payloads : Generate multiple message payloads

Inputs and Outputs

• Inputs

  • Gst Buffer with NvDsEventMsgMeta or NvDsFrameMeta/NvDsObjectMeta

• Control parameters

  • config

  • msg2p-lib

  • payload-type

  • comp-id

  • debug-payload-dir (optional)

  • multiple-payloads (optional)

  • msg2p-newapi (optional)

  • frame-interval (optional)

• Output

  • Same Gst Buffer with additional NvDsPayload metadata. This metadata contains information about the payload generated by the plugin.

Features

The following table summarizes the features of the plugin.

Gst-nvmsgconv plugin features

Feature	Description	Release
Payload in JSON format	Message payload is generated in JSON format	DS 3.0
Supports DeepStream schema specification	DeepStream schema spec implementation for messages	DS 3.0
Custom schema specification	Provision for custom schemas for messages	DS 3.0
Key-value file parsing for static properties	Read static properties of sensor/place/module in the form of key-value pair from a text file	DS 3.0
CSV file parsing for static properties	Read static properties of sensor/place/module from a CSV file	DS 3.0
DeepStream 4.0.1 minimalistic schema	Minimal variation of the DeepStream message schema	DS 4.0
New msgconv api	Supports generation of deepStream message schema payloads(full & minimal) directly using Gst buffer frame & object metadata	DS 6.0
Payload generation interval	Set frame intervals at which payloads are generated	DS 6.0

Gst Properties

The following table describes the Gst-nvmsgconv plugin’s Gst properties.

Gst-nvmsgconv plugin gst properties

Property	Meaning	Type and Range	Example / Notes	Platforms
config	Absolute pathname of a configuration file that defines static properties of various sensors, places, and modules.	String	config=msgconv_config.txt or config=msgconv_config.csv	dGPU Jetson
msg2p-lib	Absolute pathname of the library containing a custom implementation of the nvds_msg2p_* interface for custom payload generation.	String	msg2p-lib=libnvds_msgconv_custom.so	dGPU Jetson
payload-type	Type of schema payload to be generated. Possible values are: PAYLOAD_DEEPSTREAM: Payload using DeepStream schema. PAYLOAD_DEEPSTREAM_MINIMAL: Payload using minimal DeepStream schema. PAYLOAD_CUSTOM: Payload using custom schemas.	Integer, 0 to 4,294,967,295	payload-type=0 or payload-type=257	dGPU Jetson
comp-id	Component ID of the plugin from which metadata is to be processed.	Integer, 0 to 4,294,967,295	comp-id=2 Default is NvDsEventMsgMeta	dGPU Jetson
debug-payload-dir	Directory to dump payload	String	debug-payload-dir=<absolute path> Default is NULL	dGPU Jetson
multiple-payloads	Generate multiple message payloads	Boolean	multiple-payloads=1 Default is 0	dGPU Jetson
msg2p-newapi	Generate payloads using Gst buffer frame/object metadata	Boolean	msg2p-newapi=1 Default is 0	dGPU Jetson
frame-interval	Frame interval at which payload is generated	Integer, 1 to 4,294,967,295	frame-interval=25 Default is 30	dGPU Jetson

Schema Customization

This plugin can be used to implement a custom schema in two ways:

• By modifying the payload generator library: To perform a simple customization of DeepStream schema fields, modify the low-level payload generation library file sources/libs/nvmsgconv/nvmsgconv.cpp.

• By implementing the nvds_msg2p interface: If a library that implements the custom schema needs to be integrated with the DeepStream SDK, wrap the library in the nvds_msg2p interface and set the plugin’s msg2p-lib property to the library’s name. Set the payload-type property to PAYLOAD_CUSTOM.

See sources/libs/nvmsgconv/nvmsgconv.cpp for an example implementation of the nvds_msg2p interface.

Payload with Custom Objects

You can add a group of custom objects to the NvDsEventMsgMeta structure in the extMsg field and specify their size in the extMsgSize field. The meta copy (copy_func) and free (release_func) functions must handle the custom fields accordingly. The payload generator library handles some standard types of objects (Vehicle, Person, Face, etc.) and generates the payload according to the schema selected. To handle custom object types, you must modify the payload generator library nvmsgconv.cpp. See deepstream-test4 for details about adding custom objects as NVDS_EVENT_MSG_META user metadata with buffers for generating a custom payload to send to back end.