New Gst-nvstreammux
New nvstreammux is being released with DS 5.0 as an alpha feature (limited support).
Users will be able to use the new streammux by setting the environment variable “export USE_NEW_NVSTREAMMUX=yes”. By default the existing nvsteammux shall be employed.
Differences between old and new streammux with respect the GStreamer plugin properties available in both are discussed in the table below:
Old nvstreammux Properties | New nvstreammux Properties |
batch-size | batch-size |
num-surfaces-per-frame | num-surfaces-per-frame |
batched-push-timeout | N/A; Streammux config file could be used to regulate output framerate. |
width | N/A; Scaling and color conversion support disabled. |
height | N/A |
enable-padding | N/A |
gpu-id | N/A |
live-source | Deprecated |
nvbuf-memory-type | N/A |
N/A | config-file-path |
buffer-pool-size | N/A |
attach-sys-ts | Support pending and shall be added with future releases. |
The Gst-nvstreammux plugin forms a batch of frames from multiple input sources. When connecting a source to nvstreammux (the muxer), a new pad must be requested from the muxer using gst_element_get_request_pad() and the pad template "sink_%u". For more information, see link_element_to_streammux_sink_pad() in the DeepStream app source code.
The muxer forms a batched buffer of batch-size frames. (batch-size is specified using the gst object property.)
The muxer forwards the frames from that source as a part of the muxer’s output batched buffer. The frames are returned to the source when muxer gets back its output buffer.
The muxer pushes the batch downstream when the batch is filled or the batch formation timeout calculated from the overall and stream specific “fps” control configuration keys in provided streammux config file is reached. The timeout starts running when the first buffer for a new batch is collected. The default overall max and min fps for batch generation are 120 and 5 respectively.
The muxer’s default batching uses a round-robin algorithm to collect frames from the sources. It tries to collect an average of ( batch-size or num-source ) frames per batch from each source (if all sources are live and their frame rates are all the same). The number varies for each source, though, depending on the sources’ frame rates.
The muxer attaches an NvDsBatchMeta metadata structure to the output batched buffer. This meta contains information about the frames copied into the batch (e.g. source ID of the frame, original resolutions of the input frames, original buffer PTS of the input frames). The source connected to the Sink_N pad will have pad_index N in NvDsBatchMeta.
The muxer supports addition and deletion of sources at run time. When the muxer receives a buffer from a new source, it sends a GST_NVEVENT_PAD_ADDED event. When a muxer sink pad is removed, the muxer sends a GST_NVEVENT_PAD_DELETED event. Both events contains the source ID of the source being added or removed (see sources/includes/gst-nvevent.h). Downstream elements can reconfigure when they receive these events. Additionally, the muxer also sends a GST_NVEVENT_STREAM_EOS to indicate EOS from the source.
Inputs and Outputs
• Inputs
• NV12/RGBA NVMM buffers from an arbitrary number of sources
• Control Parameters
• batch-size
• num-surfaces-per-frame
• Config-file-path [config-keys detailed below]
• Output
• Batched buffer
• GstNvBatchMeta (meta containing information about individual frames in the batched buffer)
Details on Streammux config-file groups and keys:
Nvstreammux config-file | ||
Group | config-key | Description |
[property] | algorithm-type | Defines the batching algorithm; uint 1 : Round-robbin if all sources have same priority key setting. Otherwise higher priority streams will be batched until no more (or configured max) buffers from them. Default: 1 |
batch-size | The desired batch size; uint. This value will override plugin property and DS config file key “batch-size” for nvstreammux Default: 1 (or == number of sources if adaptive-batching=1) | |
overall-max-fps-n | Numerator of the desired overall muxer output max frame rate fps_n/fps_d; uint Default:120/1 | |
overall-max-fps-d | Denominator of the desired overall muxer output max frame rate fps_n/fps_d; uint | |
overall-min-fps-n | Numerator of the desired overall muxer output min frame rate fps_n/fps_d; uint Default: 5/1 | |
overall-min-fps-d | Denominator of the desired overall muxer output max frame rate fps_n/fps_d; uint | |
max-same-source-frames | Max number of any stream’s frames allowed to be muxed per output batch buffer; uint The minimum of this value and key (max-num-frames-per-batch) will be used. Default: 1 | |
adaptive-batching | Enable (1) or disable (0) adaptive batching; uint Default: 1 If enabled, batch-size is == number of sources X num-surfaces-per-frame. | |
[source-config-N] | max-fps-n | Numerator of this source’s max frame rate fps_n/fps_d; uint Default: 60/1 |
max-fps-d | Denominator of this source’s max frame rate fps_n/fps_d; uint | |
min-fps-n | Numerator of this source’s min frame rate fps_n/fps_d; uint | |
min-fps-d | Denominator of this source’s min frame rate fps_n/fps_d; uint | |
priority | The priority of this stream; uint Default: 0 (highest priority) A higher value is a lower priority. | |
max-num-frames-per-batch | Max number of this stream’s frames allowed to be muxed per output batch buffer; uint The minimum of this value and key (max-same-source-frames) will be used. | |
Features
The following table summarizes the features of the plugin.
Features of the Gst-nvstreammux plugin | ||
Feature | Description | Release |
New streammux with numerous config-keys supported in a separate mux config-file. | Introducing new streammux. | DS 5.0 |
Gst Properties
The following table describes the Gst-nvstreammux plugin’s Gst properties.
Gst-nvstreammux plugin, Gst properties | |||
Property | Meaning | Type and Range | Example Notes |
batch-size | Maximum number of frames in a batch. | Integer, 0 to 4,294,967,295 | batch-size=30 |
num-surfaces-per-frame | If non-zero, muxer scales input frames to this width. | Integer, 0 to 4,294,967,295 | num-surfaces-per-frame=1 (Default) |
config-file-path | Absolute or relative (to DS config-file location) path of mux configuration file for the Gst-nvstreammux element | String | config-file-path=config_mux_source30.txt |