Gst-nvdsanalytics¶
This plugin performs analytics on metadata attached by nvinfer
(primary detector) and nvtracker
. Analytics refer to Region of Interest (ROI) Filtering, Overcrowding Detection, Direction Detection and Line Crossing. The plugin operates in batch mode, it handles the context for each stream independently. ROI based analytics work on primary detector output, but Direction Detection and Line Crossing requires a tracker-id as previous history/state is required to do both the analytics. The configuration for the rules for analytics is achieved using a configuration file. For all the analytics calculations bottom center coordinate of bounding box of an object is being used. If bounding box is defined as (x_left, y_top, width, height) then the bottom center coordinate would be (x_left + width/2, y_top + height)
The gst-nvdsanalytics
plugin extracts the metadata from the batched buffer sent by the upstream (nvtracker/nvinfer
) element and passes it to the low-level nvdsanalytics
library. The low-level library performs analytics based on the configured rules for each feature for each of the streams, the library returns the analytics output, which is for each tracked object as well for the complete frame.
The analytics output (Frame level) is encapsulated in the NvDsAnalyticsFrameMeta
structure and is added as user meta with meta_type set to NVDS_USER_FRAME_META_NVDSANALYTICS
.The user meta is added to the frame_user_meta_list
member of NvDsFrameMeta
.
The analytics output (per detected Object) is encapsulated in the NvDsAnalyticsObjInfo
structure and is added as user meta with meta_type set to NVDS_USER_OBJ_META_NVDSANALYTICS
. The user meta is added to the obj_user_meta_list
member of NvDsObjectMeta
.
The definition of NvDsAnalyticsObjInfo
NvDsAnalyticsFrameMeta
is available in sources/includes/nvds_analytics_meta.h
header file.
For guidance on how to access user metadata, see the User/Custom Metadata addition in the NvDsBatchMeta and Tensor Metadata.
Inputs and Outputs¶
Inputs
GStreamer buffer containing NV12/RGBA frame(s) with attached metadata
Control Parameters
enable: True Enables the plugin, False sets it in pass-through mode
config-file: Configuration file which consist of the analytics rules
Output
Original NV12/RGBA batched buffer
NvDsAnalyticsFrameMeta
(meta containing information about individual frames in the batched buffer)NvDsAnalyticsObjInfo
(meta containing information about individual objects in the individual frames of batched buffer)
Features¶
The following table summarizes the features of the plugin.
Feature |
Description |
Release |
---|---|---|
ROI Filtering |
This feature detects presence of objects in the ROI, the result is attached as metadata for every object, as well as total count of objects in ROI per frame. |
DS 5.0 |
Overcrowding Detection |
This feature detects presence of objects in the ROI, the result is attached as metadata for every object, as well as total count of objects in ROI per frame along with overcrowding status, i.e. no of objects in ROI exceed a pre-configured threshold. |
DS 5.0 |
Direction Detection |
Checks using history of position of object and the current frame information if it is following a preconfigured direction. The result is attached as metadata for every object. |
DS 5.0 |
Line Crossing |
This feature checks if object is following a preconfigured direction for the virtual line and if it has crossed the virtual line. The result is attached to object when Line crossing also frame meta where cumulative count along with per frame count of line crossing is attached. |
DS 5.0 |
Gst Properties¶
The following table describes the Gst properties of the Gst-nvdsanalytics plugin.
Property |
Meaning |
Type and Range |
Example Notes |
---|---|---|---|
unique-id |
Unique identification unsigned integer value for element to identify the output attached by the element |
Integer, 0 to 4,294,967,295 |
unique-id=17 |
enable |
True Enables the plugin, False sets it in pass-through mode |
Boolean |
enable=1 |
config-file |
Configuration file which consist of the analytics rules |
String |
config-file=config_nvdsanalytics.txt |
Configuration File Parameters¶
The configuration file is located at sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt.
The Gst-nvdsanalytics configuration file uses a “Key File” format described in: https://specifications.freedesktop.org/desktop-entry-spec/latest
The
[property]
group configures the general behavior of the plugin. It is the only mandatory group.The
[roi-filtering-stream-<n>]
group configures ROI filtering rules parameters for stream specified by <n>. For example,classes.[roi-filtering-stream-0]
configures ROI filtering rules for stream ID 0.The
[overcrowding-<n>]
group configures Overcrowding parameters for a stream specified by` <n>`.The
[direction-detection-<n>]
group configures Direction Detection parameters for a stream specified by <n>.The
[line-crossing-<n>]
group configures Line Crossing parameters for a stream specified by <n>.
The following table respectively describe the keys supported for [property]
groups and [<feature>-stream-<n>]
groups.
Property |
Applicable Groups |
Meaning |
Type and Range |
Example Notes |
---|---|---|---|---|
enable |
[property] | [<feature>-stream-<n>] |
Common property to enable or disable the plugin or feature |
Boolean |
enable=1 |
config-width |
[property] |
Width for which the analytics rules are configured |
Integer, >0 |
config-width=1280 |
config-height |
[property] |
Height for which the analytics rules are configured |
Integer, >0 |
config-height=720 |
osd-mode |
[property] |
0: Disables display of any lines, ROIS and text 1: Display only lines, rois and static text i.e. labels 2: Display all info from 1 in addition, information for all the events |
Integer0,1,2 |
osd-mode=2 |
display-font-size |
[property] |
Set OSD font size for the analytics data |
Integer > 0 |
display-font-size=12 |
roi-<label> |
[roi-filtering-stream-<n>] | [overcrowding-stream-<n>] |
The ROI can be specified using roi key followed by its corresponding label (roi-<label>) the value consist of semi-colon delimited coordinates of the polygon (roi-<label>=x1;y1;x2;y2;x3;y3;x4;y4;x5;y5) . The user can specify a simple polygon with > 3 points. The points should be sequentially specified, as the relative position of points is required to check object in polygon. |
String |
roi-RF= 295;643;579;634;642;913;56;828 roi-OC= 295;643;579;634;642;913;56;828 |
inverse-roi |
[roi-filtering-stream-<n>] |
If the flag set true, it indicates the object outside the configured ROI be attached with the ROI label, and count the objects outside the ROI and send as frame level output |
Boolean |
inverse-roi=0 |
class-id |
[<feature>-stream-<n>] |
The class ID separated by semi-colon for which analytic rules should be applied. | -1 indicate that the rules are to be applied to all the classes |
String |
class-id=0;1;2 or class-id=-1 |
object-threshold |
[overcrowding-stream-<n>] |
The total count after which overcrowding flag should be indicated in the frame metadata. |
Integer, >=0 |
object-threshold=10 |
direction-<label> |
[direction-detection-stream-<n>] |
The value indicates the direction (x1;y1;x2;y2) which corresponds to the configured <label>. When object is moving in configured direction, the label of the particular rule is attached to the object as user metadata |
String |
direction-South=284;840;360;662 |
line-crossing-<label> |
[line-crossing-stream-<n>] |
The value indicates 2 coordinates of direction followed by 2 coordinates of virtual line. These coordinates are delimited using semi-colon (line-crossing-<label>=x1d;y1d;x2d;y2d;x1c;y1c;x2c;y2c). The direction indicates the direction of object in which it should be moving that can be considered for counting, so the object should be in direction x1d,y1d to x2d,y2d, which indicates a direction vector. The line crossing coordinates x1c,y1c and x2c,y2c indicate a virtual line which when an object following a configured direction crosses is then accounted for line crossing. <label> is string identification of the rule. Multiple line-crossings can be defined for a single stream |
String |
line-crossing-Exit=789;672;1084;900;851;773;1203;732 |
extended |
[line-crossing-stream-<n>] |
Boolean parameter to check crossings only on the configured line or for the extended line covering the video frame. If extended is 0, the line crossing only the configured line would be counted |
Boolean |
extended=0 |
mode |
[line-crossing-stream-<n>] [direction-detection-stream-<n>] |
Parameter to allow user configuration of adherence of object direction with configured direction. Possible options strict, balanced, loose. Operate as per name suggestion, where loose will just check if the object has crossed the line and would need the object just to be in same direction with very high tolerance |
String |
mode=loose
mode=strict
mode=balanced
|
Note
The config-width, config-height, indicate the resolution at which analytics rules have been configured. If the runtime stream resolution is different than the configuration resolution. The plugin handles the resolution change and scales the rules for the runtime resolution. The configuration rules are not present in absolute pixel values.
The following pipelines demonstrate plugin’s usage.
For dGPU:
gst-launch-1.0 filesrc location=streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 live-source=1 ! nvinfer config-file-path= sources/apps/sample_apps/ deepstream-nvdsanalytics-test/nvdsanalytics_pgie_config.txt ! nvtracker ll-lib-file = /opt/nvidia/deepstream/deepstream/lib/libnvds_nvdcf.so ll-config-file= tracker_config.yml tracker-width=640 tracker-height=384 ! nvdsanalytics config-file= sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt ! nvmultistreamtiler ! nvvideoconvert ! nvdsosd ! nveglglessink
For Jetson:
gst-launch-1.0 filesrc location=streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 live-source=1 ! nvinfer config-file-path= sources/apps/sample_apps/ deepstream-nvdsanalytics-test/nvdsanalytics_pgie_config.txt ! nvtracker ll-lib-file = /opt/nvidia/deepstream/deepstream/lib/libnvds_nvdcf.so ll-config-file= tracker_config.yml tracker-width=640 tracker-height=384 ! nvdsanalytics config-file= sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt ! nvmultistreamtiler ! nvvideoconvert ! nvdsosd ! nv3dsink