DeepStream With REST API Sever (Alpha)

DeepStream application supports runtime parameter configuration for certain components with the help of REST APIs. DeepStream application should run as a server for this feature.

Rest Server support is enabled for DeepStream-6.2 release and onwards. Currently it is in alpha development stage. REST Server library integrates the HTTP server using the “Civetweb” OSS library to process the REST API endpoints. REST Server library implements various component specific endpoints handling. Also, REST API Payload Parser is integrated using the “jsoncpp” OSS library to parse JSON format payload. New nvmultiurisrcbin implements callback APIs for runtime parameter configuration of the supported components in the native DeepStream gstreamer pipeline. Upon receiving the HTTP POST request for the REST API, relevant callbacks implemented with the nvmultiurisrcbin creates new custom gstreamer events which gets injected into the native gstreamer pipeline. Applicable gstreamer component intercepts the custom event to runtime configure the relevant component in the native gstreamer pipeline. A sample app deepstream-server-app is also provided at /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-server for reference. These interfaces sources are available in the DeepStream SDK package. This should provide users with readily available infrastructure to define and implement their own custom REST APIs support for various Deepstream components (Please refer “How to add custom REST API support” section below). REST server is instantiated by default inside nvmultiurisrcbin plugin.

DeepStream Rest Server block diagram

Refer to Gst-nvmultiurisrcbin (Alpha) for “nvmultiurisrcbin” details.

Features

The sample deepstream-server application can be used to demonstrate usage of REST APIs to configure DeepStream component parameters at runtime. Currently supported REST APIs to configure the DeepStream pipeline as mentioned below:

The following table summarizes the supported REST APIs

Rest API support

Group/Component name

Parameter update

Stream specific

Stream addition and removal

nvdspreprocess

Update ROI for preprocess

nvv4l2decoder

“drop-frame-interval” property update

nvdsinfer

“interval” property update

nvds_rest_server

nvds_rest_server is a custom DeepStream library open sourced with DeepStreamSDK package at /opt/nvidia/deepstream/deepstream/sources/libs/nvds_rest_server/. Users are encouraged to use this library as-is or make modifications to support custom use cases. This library could be integrated into a Gstreamer plugin, GStreamer bin, or a DeepStream application. With DS 6.2 release, nvds_rest_server library is integrated with Gst-nvmultiurisrcbin (Alpha) to support runtime sensor addition and removal over REST API.

deepstream-server-app

The deepstream-server-app sample application is provided at /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-server. The reference application demonstrates REST API usage to configure DeepStream plugin parameters at runtime using nvmultiurisrcbin. Refer to the README at /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-server/README for application usage, payload schema and REST API endpoints details.

Sensor provisioning with deepstream-test5-app

The DeepStream config file parsing reference apps like deepstream-test5-app support Sensor provisioning (runtime stream add/remove). For more details and sample config file to refer, please follow documentation here.

How to add custom REST API support

Users should follow the below sections. Each section explains detailed steps to implement new custom REST API support.

  1. REST API endpoints implementation

  2. Custom event generation

  3. Callback implementation for REST API endpoints

REST API endpoints implementation

  1. Define any component/group specific custom REST API endpoints.

  2. Register custom REST API endpoints.

    For 1 & 2, refer below source code snippet in /opt/nvidia/deepstream/deepstream/sources/libs/nvds_rest_server/nvds_rest_server.cpp:

m_func["/roi/update"] =
  [roi_cb] (const Json::Value & req_info, const Json::Value & in,
  Json::Value & out, struct mg_connection * conn) {
     return handleUpdateROI (req_info, in, out, conn, roi_cb);
};

  1. Define and implement the Handler function corresponding to the new custom REST API.

    Refer to the below source code snippet in /opt/nvidia/deepstream/deepstream/sources/libs/nvds_rest_server/nvds_rest_server.cpp:

NvDsErrorCode
handleUpdateROI (const Json::Value & req_info, const Json::Value & in,
  Json::Value & response, struct mg_connection *conn,
  std::function < void (NvDsRoiInfo * roi_ctx, void *ctx) > roi_cb);

Refer below Handler Function Implementation Details section for more details.

  1. Register the Handler function for the new custom REST API with Civerweb’s “httpServerHandler->addHandler” interface.

Refer to the below source code snippet in /opt/nvidia/deepstream/deepstream/sources/libs/nvds_rest_server/nvds_rest_server.cpp:

httpServerHandler->addHandler (it.first, new RequestHandler (it.first,
        it.second));

  1. Define callback function to be used by REST server for the corresponding new custom REST API. The user needs to add callback function pointer as member of the struct NvDsServerCallbacks.

    Refer to the below source code snippet in /opt/nvidia/deepstream/deepstream/sources/libs/nvds_rest_server/nvds_rest_server.h:

typedef struct NvDsServerCallbacks
{
  std::function < void (NvDsRoiInfo * roi_info, void *ctx) > roi_cb;
  std::function < void (NvDsDecInfo * dec_info, void *ctx) > dec_cb;
  std::function < void (NvDsStreamInfo * stream_info, void *ctx) > stream_cb;
  std::function < void (NvDsInferInfo * infer_info, void *ctx) > infer_cb;
} NvDsServerCallbacks;

Handler Function Implementation Details

  1. Define and implement a parser function to parse payload data received with the HTTP request for the new custom REST API. User may define new custom data structure (with nvds_rest_server.h) to hold the parsed payload data.

    Refer below source code snippet in /opt/nvidia/deepstream/deepstream/sources/libs/nvds_rest_server/nvds_roi_parse.cpp

bool
nvds_rest_roi_parse (const Json::Value & in, NvDsRoiInfo * roi_info)

  1. Call the defined callback providing input the payload parsed data and custom context pointer (if any).

  2. Based on the callback function return status received, create a JSON response to be sent for the received HTTP request.

Refer to the below source code snippet in /opt/nvidia/deepstream/deepstream/sources/libs/nvds_rest_server/nvds_rest_server.cpp:

  if (nvds_rest_roi_parse (in, &roi_info) && (roi_cb)) {
    roi_cb (&roi_info, &custom_ctx);
    if (roi_info.status == ROI_UPDATE_SUCCESS)
      res_info.status = "ROI_UPDATE_SUCCESS";
    else
      res_info.status = "ROI_UPDATE_FAIL";
  } else {
    res_info.status = "ROI_UPDATE_FAIL";
  }
  res_info.reason = "NA";
  response["status"] = res_info.status;
  response["reason"] = res_info.reason;
}

Custom event generation

  1. Define and implement new custom gst-event applicable for specific “element or group of elements” corresponding to handling of the new custom REST API.

Refer to the below source code snippet in /opt/nvidia/deepstream/deepstream/sources/libs/gstnvdscustomhelper/gst-nvcustomevent.c

GstEvent *
gst_nvevent_new_roi_update (gchar* stream_id, guint roi_count, RoiDimension *roi_dim)

  1. This custom gst-event to be injected into the native gstreamer pipeline by the gstnvdsmulturisrcbin.

Callback implementation for REST API endpoints

  1. Implement the callback function (as registered with the struct NvDsServerCallbacks) defined for the new custom REST API.

    Refer to the below source code snippet of /opt/nvidia/deepstream/deepstream-6.2/sources/libs/gst-nvmultiurisrcbin/gstdsnvmultiurisrcbin.cpp

static void s_roi_api_impl(NvDsRoiInfo *roi_info, void *ctx)

  1. This callback function injects the new custom gst-event into the pipeline to be handled by corresponding gstreamer element or group of elements. Applicable gstreamer element or group of elements need to intercept this new custom gst-event at sink pad event handler and apply received configurations to the applicable element at runtime.

    Refer to the below source code snippet of /opt/nvidia/deepstream/deepstream-6.2/sources/libs/gst-nvmultiurisrcbin/gstdsnvmultiurisrcbin.cpp

GstEvent *nvevent = gst_nvevent_new_roi_update((char*)roi_info->stream_id.c_str(), roi_info->roi_count, roi_dim);
if (!gst_pad_push_event ((GstPad *)(nvmultiurisrcbin->bin_src_pad), nvevent)) {
  g_print("[WARN] ROI UPDATE event not pushed downstream.. !! \n");
  roi_info->status = ROI_UPDATE_FAIL;
} else {
  roi_info->status = ROI_UPDATE_SUCCESS;
}

  1. In case the applicable element is already a part of the gstdsnvmultiurisrbin (e.g. decoder) then no such custom gst-events are required to be injected into the pipeline as all runtime configuration for the applicable gst-element can be handled from inside the gstdsnvmultiurisrcbin.

    Refer to the below source code snippet of /opt/nvidia/deepstream/deepstream-6.2/sources/libs/gst-nvmultiurisrcbin/gstdsnvmultiurisrcbin.cpp

if (!set_nvuribin_dec_prop(nvmultiurisrcbin->nvmultiurisrcbinCreator,sourceId, dec_info->drop_frame_interval)) {
g_print("[WARN] drop-frame-interval not set on decoder .. !! \n");
dec_info->status = DROP_FRAME_INTERVAL_UPDATE_FAIL;
} else {
  dec_info->status = DROP_FRAME_INTERVAL_UPDATE_SUCCESS;
}

Limitations

  1. REST API feature supported on X86 only.

  2. REST API feature is validated with HTTP. HTTPS support is not yet enabled, however users can add HTTPS support by enhancing corresponding sources in DeepStream SDK.