What is a Deepstream Service Maker Plugin

A plugin encapsulates a specialized factory, enabling the creation of user-implemented custom objects through a standardized factory API. Users have the flexibility to define a factory in a plugin with descriptive metadata, including name, version, descriptions, and more, for their custom objects. By loading the plugin, any application gains the ability to create instances of the custom object through the common factory API and facilitates the seamless reuse of user-defined custom objects (e.g., object counter, latency measurement, etc) across multiple applications.

Deepstream Service Maker Module

Getting Started with Your Factory and Plugin

Prior to encapsulating a factory, it is essential to come up with a custom object that the factory will create. As an illustrative example, let’s consider a simple latency measurement probe. By implementing the IBufferObserver interface for a BufferProbe, we tailor it to serve as a latency measurement tool for video frames:

class MeasureLatency : public BufferProbe::IBufferObserver {
  public:
    virtual probeReturn handleBuffer(BufferProbe& probe, const Buffer& buffer) {
        auto latency_info = buffer.measureLatency();
        for (auto& latency : latency_info) {
            cout << "Source id = " << latency.source_id
                << " Frame_num = " << latency.frame_num
                << " Frame latency = " << latency.latency << " (ms)"
                << endl;
        }
        return probeReturn::Probe_Ok;
    }
};

Next, you need to encapsulate a factory for this latency measurement probe within a plugin. To achieve this, we can employ macros in the plugin.h file to define both the factory and the plugin, respectively:

DS_CUSTOM_PLUGIN_DEFINE(measure_latency_probe, "Custom probe to add measure Latency", "0.1", "Proprietary")

// define a factory that creates a BufferProbe instance with a MeasureLatency implementation
DS_CUSTOM_FACTORY_DEFINE(
"measure_latency_probe",
"Latency measurement calculating custom probe factory",
"probe",
"this is a latency measurement custom probe factory",
"NVIDIA",
BufferProbe,
MeasureLatency
)

To build the plugin, you need to create a straightforward CMakeLists.txt file with the following content:

cmake_minimum_required(VERSION 3.16)

project(Sample)
find_package(nvds_c++ REQUIRED PATHS /opt/nvidia/deepstream/deepstream/service-maker/cmake)

add_library(measure_latency_probe SHARED measure_latency_probe.cpp)
target_link_libraries(measure_latency_probe PRIVATE nvds_service_maker nvds_service_maker_utils)

$ mkdir build && cd build && cmake .. && make

Upon successful completion of the plugin build process, you can seamlessly utilize our latency measurement-customized BufferProbe in any application:

pipeline.attach("decoder", "measurelatencyprobe", "my probe", "src")

Incorporating properties to a plugin

To further increase the versatility of factory reuse, we can introduce property specification for the objects from the factory. Macros for adding property specification to the factory can be utilized as follows:

DS_CUSTOM_FACTORY_DEFINE_PARAMS_BEGIN(probe_param_spec)
DS_CUSTOM_FACTORY_DEFINE_PARAM(source-id, integer, "source-id", "sourceid",  0)
DS_CUSTOM_FACTORY_DEFINE_PARAMS_END

Within the latency measurement implementation, the calculation of frame latency can be confined to a specified stream:

class MeasureLatency : public BufferProbe::IBufferObserver {
  public:
    virtual probeReturn handleBuffer(BufferProbe& probe, const Buffer& buffer) {
        int id = 0;
        probe.getProperty("source-size", id);
        auto latency_info = buffer.measureLatency();
        for (auto& latency : latency_info) {
            if (latency.source_id == id) {
               cout << "Source id = " << latency.source_id
                    << " Frame_num = " << latency.frame_num
                    << " Frame latency = " << latency.latency << " (ms)"
                    << endl;
            }
        }
        return probeReturn::Probe_Ok;
    }
};

In the application, you would have the option to specify the precise source ID for latency measurement:

pipeline.attach("decoder", "measurelatencyprobe", "my probe", "src", "source-id", 1)

Sample Plugins

Sample plugins can be found from the Deepstream installation directory /opt/nvidia/deepstream/deepstream/service-maker/sources/modules, and with the following commands user can build them:

$ cmake /opt/nvidia/deepstream/deepstream/service-maker/sources/modules/sample_video_probe && make

Message Metadata Generator

The Message Metadata Generator is a sample plugin that creates a buffer probe to generate event message metadata. This metadata can be utilized by downstream nvmsgconv to create JSON payloads for transmission to message brokers like Kafka.

Registered name of the plugin is “add_message_meta_probe”.

KITTI Dump Plugin

The KITTI Dump Plugin is a sample plugin that creates a buffer probe to dump the object metatdata there into a txt file in kitti format. The registered name of the plugin is “ktti_dump_probe”.

Following parameters are supported:

  • kitti-dir: the path for saving kitti files (default: /tmp/kitti).

Framerate Measurement Plugin

The Framerate Measurement Plugin is a sample plugin that creates a framerate measurement probe which measures and prints the current frame rate. The registered name of the plugin is “measure_fps_probe”.

Latency Measurement Plugin

The Latency Measurement Plugin is a sample plugin that creates a latency measurement probe which measures and prints the current frame latency. The registered name of the plugin is “measure_latency_probe”.

Sample Video Probe Plugin

This plugin creates a sample video probe which constructs the display metadata for downstream osd plugin to show the object counter. The registered name of the plugin is “sample_video_probe”. The following parameters are supported for customizing the created buffer probe object:

  • font-size: an integer to specify the size of the font.

Sample Signal Handler Plugin

This plugin creates a sample signal handlers that responds on the “model-updated” signal from nvinfer plugin. As a sample it prints out the message whenever the model is updated. The registered name of the plugin is “sample_signal_handler”.

Sample Video Feeder Plugin

This plugin creates a sample video feeder can be attached to an “appsrc”, thus injects raw video data to the pipeline from a file. The registered name of the plugin is “sample_video_feeder”. The following parameters are supported for customizing the created buffer probe object:

  • location: a string to specifies the file as the source of video data

  • frame-width: width of the video frame

  • frame-height: height of the video frame

  • format: format of the video frame, which can be one among RGBA, I420 and NV12

  • use-gpu-memory: True for copying data into a GPU memory

  • use-external-memory: True for allocating the memory outside the pipeline.

Sample Video Receiver Plugin

This plugin creates a sample video receiver to demonstrate how to retrieve the buffer data from an “appsink”. The registered name of the plugin is “sample_video_receiver”.

Smart Recording Action Plugin

This plugin creates a signal emitter object which can attach to “nvurisrcbin” to trigger smart record and pause it. As a sample plugin it interacts with a remote Kafka server and controls the smart recording based on the messages received from the Kafka server.

The registered name of the plugin is “smart_recording_action”. The following parameters are supported for customizing the created buffer probe object:

  • proto-lib: the path to the shared library that implements the device/cloud communication protocol

  • conn-str: string for the connection in the format of ‘ip;port’

  • proto-config-file: path to the config file of the communication protocol

  • msgconv-config-file: message converter config file path

  • topic-list: list of topics to subscribe

Smart Recording Signal Plugin

This plugin creates a sample smart recording signal handler which responds on “sr-done” signal from “nvurisrcbin” with a brief print. The registered name of the plugin is “smart_recording_signal”.

Plugin Search Path

By default the plugins will be searched from the installation path /opt/nvidia/deepstream/deepstream/service-maker/sources/modules. However, users can specify their path by defining them in the NVDS_MODULE_PATH environment variable, and multiple paths can be separated by “:”.