> This page is for version 9.2 (default). > For other versions, use one of these documentation indexes: > - 9.2 (default): https://docs.nvidia.com/deepstream/9.2/llms.txt > - Archives: https://docs.nvidia.com/deepstream/archives/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/deepstream/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/deepstream/_mcp/server. # Gst-nvmodelmux The Gst-nvmodelmux plugin is a DeepStream GStreamer bin that routes dynamic input streams to per-stream primary and optional shadow inference models behind a single batched input and a single batched output. It supports runtime model and stream-routing control through the DeepStream REST/control-message interfaces, letting a pipeline swap or A/B compare inference models per stream without reconstructing the pipeline. Streams beyond a model instance's per-model batch cap can automatically shard onto additional same-model instances (model-sharding). ## Inputs and Outputs * Inputs * GStreamer buffer containing NvDsBatchMeta (batched input, one or more streams per batch) * Control parameters * enable, config-file-path, batch-size, per-model-batch-size, unified-batch, wait-for-default-models, compact-idle-models-across-pools, primary-model, shadow-model, model-sharding * Runtime model and stream-routing control via the DeepStream REST/control-message action-signal interface * Output * GStreamer buffer containing NvDsBatchMeta with inference output from the routed primary (and, if configured, shadow) model for each stream ## Features The following table summarizes the features of the plugin. **Gst-nvmodelmux plugin features** | Feature | Description | Release | | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | Per-stream model routing | Routes each stream to its own configured primary (and optional shadow) inference model behind one batched input/output | DS 9.2 | | Primary/shadow A/B comparison | Runs a primary and an optional shadow model per stream and can combine both outputs into one batched buffer (unified-batch) for A/B comparison | DS 9.2 | | Runtime model and routing control | Supports runtime model swap and stream-routing changes through the DeepStream REST/control-message interface without pipeline reconstruction | DS 9.2 | | Automatic model sharding | Automatically distributes streams beyond a model instance's per-model batch cap onto additional same-model shard instances | DS 9.2 | ## Gst Properties The following table describes the Gst properties of the Gst-nvmodelmux plugin. **Gst-nvmodelmux plugin gst properties** | Property | Meaning | Type and Range | Example Notes | | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | ----------------------------------------------------- | | enable | Runs inference (TRUE) or acts as a transparent passthrough (FALSE) | Boolean | enable=1 | | config-file-path | Multimodel config file ("\[multimodel]" defaults/behavior plus per-sensor "\[stream-model-\]" bindings) parsed by the plugin; individual properties override values from it | String | config-file-path=config_modelmux.txt | | batch-size | Maximum number of concurrent streams, sizing every internal mux; 0 inherits from the config file/default and should match the upstream streammux batch | Integer, 0 to MM_MAX_BATCH | batch-size=0 | | per-model-batch-size | nvinfer batch size for a single model instance (0 uses batch-size); streams beyond this auto-shard onto additional same-model instances | Integer, 0 to MM_MAX_BATCH | per-model-batch-size=0 | | unified-batch | Combines primary and shadow outputs into one batched buffer for an A/B view | Boolean | unified-batch=0 | | wait-for-default-models | Blocks the pipeline's transition to PLAYING until the configured default model(s) are warmed, so defaults run inference from the first frame (mirrors nvinfer); FALSE warms them asynchronously and streams passthrough until ready | Boolean | wait-for-default-models=0 | | compact-idle-models-across-pools | When a model goes idle as a lone redundant warm copy, moves it out of its role pool (primary/shadow) into a shared limbo pool so a later request from either role reuses that warm copy, keeping fewer warm copies overall; applies only when no twin already exists, and the configured default model is never moved | Boolean | compact-idle-models-across-pools=0 | | primary-model | Default PRIMARY model as "name;version;config-file" (config-file required; name defaults to its basename, version optional); required to enable inference | String | primary-model=trafficcamnet;;config_infer_primary.txt | | shadow-model | Optional default SHADOW model (A/B) as "name;version;config-file"; unset means no shadow | String | shadow-model=car;;config_infer_secondary_car.txt | | model-sharding | Enables automatic sharding of streams across model instances. TRUE (default): overflow streams beyond the per-model batch cap are distributed onto new same-model shard instances. FALSE: streams exceeding the cap are silently ignored, matching stock nvinfer behavior | Boolean | model-sharding=1 |