Skip to navigation
DeepStream Plugin GuidePlugins

Gst-nvmodelmux

View as Markdown

The Gst-nvmodelmux plugin is a DeepStream GStreamer bin that routes dynamic input streams to per-stream primary and optional shadow inference models behind a single batched input and a single batched output. It supports runtime model and stream-routing control through the DeepStream REST/control-message interfaces, letting a pipeline swap or A/B compare inference models per stream without reconstructing the pipeline. Streams beyond a model instance’s per-model batch cap can automatically shard onto additional same-model instances (model-sharding).

Inputs and Outputs

  • Inputs

    • GStreamer buffer containing NvDsBatchMeta (batched input, one or more streams per batch)

  • Control parameters

    • enable, config-file-path, batch-size, per-model-batch-size, unified-batch, wait-for-default-models, compact-idle-models-across-pools, primary-model, shadow-model, model-sharding
    • Runtime model and stream-routing control via the DeepStream REST/control-message action-signal interface

  • Output

    • GStreamer buffer containing NvDsBatchMeta with inference output from the routed primary (and, if configured, shadow) model for each stream

Features

The following table summarizes the features of the plugin.

Gst-nvmodelmux plugin features

FeatureDescriptionRelease
Per-stream model routingRoutes each stream to its own configured primary (and optional shadow) inference model behind one batched input/outputDS 9.2
Primary/shadow A/B comparisonRuns a primary and an optional shadow model per stream and can combine both outputs into one batched buffer (unified-batch) for A/B comparisonDS 9.2
Runtime model and routing controlSupports runtime model swap and stream-routing changes through the DeepStream REST/control-message interface without pipeline reconstructionDS 9.2
Automatic model shardingAutomatically distributes streams beyond a model instance’s per-model batch cap onto additional same-model shard instancesDS 9.2

Gst Properties

The following table describes the Gst properties of the Gst-nvmodelmux plugin.

Gst-nvmodelmux plugin gst properties

PropertyMeaningType and RangeExample Notes
enableRuns inference (TRUE) or acts as a transparent passthrough (FALSE)Booleanenable=1
config-file-pathMultimodel config file (“[multimodel]” defaults/behavior plus per-sensor “[stream-model-<id>]” bindings) parsed by the plugin; individual properties override values from itStringconfig-file-path=config_modelmux.txt
batch-sizeMaximum number of concurrent streams, sizing every internal mux; 0 inherits from the config file/default and should match the upstream streammux batchInteger,
0 to MM_MAX_BATCH		batch-size=0
per-model-batch-sizenvinfer batch size for a single model instance (0 uses batch-size); streams beyond this auto-shard onto additional same-model instancesInteger,
0 to MM_MAX_BATCH		per-model-batch-size=0
unified-batchCombines primary and shadow outputs into one batched buffer for an A/B viewBooleanunified-batch=0
wait-for-default-modelsBlocks the pipeline’s transition to PLAYING until the configured default model(s) are warmed, so defaults run inference from the first frame (mirrors nvinfer); FALSE warms them asynchronously and streams passthrough until readyBooleanwait-for-default-models=0
compact-idle-models-across-poolsWhen a model goes idle as a lone redundant warm copy, moves it out of its role pool (primary/shadow) into a shared limbo pool so a later request from either role reuses that warm copy, keeping fewer warm copies overall; applies only when no twin already exists, and the configured default model is never movedBooleancompact-idle-models-across-pools=0
primary-modelDefault PRIMARY model as “name;version;config-file” (config-file required; name defaults to its basename, version optional); required to enable inferenceStringprimary-model=trafficcamnet;;config_infer_primary.txt
shadow-modelOptional default SHADOW model (A/B) as “name;version;config-file”; unset means no shadowStringshadow-model=car;;config_infer_secondary_car.txt
model-shardingEnables automatic sharding of streams across model instances. TRUE (default): overflow streams beyond the per-model batch cap are distributed onto new same-model shard instances. FALSE: streams exceeding the cap are silently ignored, matching stock nvinfer behaviorBooleanmodel-sharding=1