> This page is for version 9.2 (default). > For other versions, use one of these documentation indexes: > - 9.2 (default): https://docs.nvidia.com/deepstream/9.2/llms.txt > - Archives: https://docs.nvidia.com/deepstream/archives/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/deepstream/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/deepstream/_mcp/server. # Gst-nvdspreprocess The Gst-nvdspreprocess plugin is a customizable plugin which provides a custom library interface for preprocessing on input streams. Each stream can have its own preprocessing requirements. (e.g. per stream ROIs - Region of Interests processing) Streams with same preprocessing requirements are grouped and processed together. These requirements can be specified via a config file. Refer to section [Gst-nvdspreprocess File Configuration Specifications](/text/DS_plugin_gst-nvdspreprocess#gst-nvdspreprocess-file-configuration-specifications) below for more details. Default plugin implementation provides two functionalities. 1. Streams with predefined ROIs (Region of Interests) are scaled and format converted as per the network requirements for inference. Per stream ROIs are specified in the config file. 2. It prepares a raw tensor from the scaled & converted ROIs. It is passed to the downstream plugins via user metadata. Downstream plugins can access this tensor for inference. The default custom library (nvdspreprocess_lib) provided with the plugin implements these functionalities. It contains the group transformation (scaling and conversion functionality per group) and tensor preparation functions for 2d convolution NCHW/NHWC models. User can implement its own library with group transformation functions and tensor preparation function as per the requirements. This plugin can operate in two modes PGIE and SGIE. Preprocess with PGIE mode is used to process the given ROI/Frame on which we want to perform primary inferencing. Preprocess with SGIE mode is used to process the detected objects within the given ROI/Frame on which we want to perform secondary inferencing. ![Gst-nvdspreprocess](/deepstream/_fern-img/5eb56b3590fe61f107d345debf847bc0de256485dba92f74ec97c71152bfbe7d.webp) ## Inputs and Outputs * Inputs * Input Video Gst Buffers * Metadata (NvDsBatchMeta) * Control parameters * config-file * Output * Output Video Gst Buffers * Metadata (NvDsBatchMeta) + User Metadata at batch level (NvDsPreProcessBatchMeta) ## Features The following table summarizes the features of the plugin. **Gst-nvdspreprocess features** | Feature | Description | Release | | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | ------- | | Per stream/group of streams ROI processing | Process on the pre-defined ROIs having same processing algorithm inside a group | DS 6.0 | | Processing on full frames/ROIs | Enable process-on-roi inside \[group-\*] to process on ROIs | DS 6.0 | | Custom library interface | Custom functionality using group transformation and tensor preparation functions | DS 6.0 | | Custom group transformation function | For each group separate custom transformation functions can be provided from custom lib (e.g. scale & conversion) | DS 6.0 | | Custom tensor preparation function | Custom tensor preparation function from custom library to prepare raw tensor from already transformed ROIs/full-frames | DS 6.0 | | Attach user meta at batch level | User meta (NvDsPreProcessBatchMeta) having raw tensor and Scaled & converted ROIs | DS 6.0 | | Support for SGIE Mode | Supports processing on objects detected by PGIE model | DS 6.2 | ## Custom library Interfaces * custom_transform => An Interface to implement custom transformation for different group of streams. Default implementation provides group transformation functionalities of ROIs specified in the config fie. * custom_tensor_function => An interface to provide custom tensor preparation function. Default implementation prepares raw tensor from transformed ROIs ## Gst-nvdspreprocess File Configuration Specifications The Gst-nvdspreprocess configuration file uses a “Key File” format described in [https://specifications.freedesktop.org/desktop-entry-spec/latest](https://specifications.freedesktop.org/desktop-entry-spec/latest). Refer to `config_preprocess.txt` at `/opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt`. The `[property]` group configures the general behavior of the plugin. The `[group-]` group configures ROIs/full-frames for a group of streams with `src-ids` and `custom-input-transformation-function` from custom lib. The `[user-configs]` group configures parameters required by the custom library, which is passed on to custom lib through a map of `` as key-value pair. custom lib needs to parse the values accordingly. The following two tables describes the keys supported for `[property]` groups and `[group-]` groups respectively. **Gst-nvdspreprocess property Group Supported Keys** | Property | Meaning | Type and Range | Example | | ---------------------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | | enable | If set then to enable the plugin else in passthrough mode | Boolean | enable=1 | | unique-id | Uniquely identify the metadata generated by this element | Integer, \| 0 to 4,294,967,295 | unique-id=1 | | gpu-id | Device ID of GPU to use for pre-processing (dGPU only) | Integer,0-4,294,967,295 | gpu-id=1 | | process-on-frame | Preprocessing Modes 1=PGIE Mode 0=SGIE Mode | Boolean | process-on-frame=1 | | target-unique-ids | List of component gie-id for which tensor is prepared | Semicolon delimited integer(gie-ids) array of values >=0 | target-unique-ids=3;4;5 | | operate-on-gie-id | Unique gie-id whose metadata is to be preprocessed (ignored if process-on-frame enabled) | Integer, 0 to 4,294,967,295 | operate-on-gie-id=1 | | network-input-order | Order of the network input layer | Integer 0=NCHW 1=NHWC 2=CUSTOM | network-input-order=0 | | network-input-shape | Tensor shape of network input layer as per network-input-order | Semicolon delimited integer array of values > 0 | network-input-shape=60;3;368;640 | | maintain-aspect-ratio | If set then maintain the aspect ratio while scaling | Boolean | maintain-aspect-ratio | | symmetric-padding | If set then pad symmetrically (ignored if maintain-aspect-ratio disabled) | Boolean | symmetric-padding | | processing-width | Width at which ROIs scaled | Integer>0 | processing-width=640 | | processing-height | Height at which ROIs scaled | Integer>0 | processing-height=368 | | scaling-buf-pool-size | Size of scaling buffer pool | Integer>0 | scaling-buf-pool-size=6 | | tensor-buf-pool-size | Size of tensor buffer pool | Integer>0 | tensor-buf-pool-size=6 | | network-color-format | Color format of model | Integer 0=RGB 1=BGR 2=GRAY | network-color-format=0 | | tensor-data-type | Data format for inference | Integer 0=FP32 1=UINT8 2=INT8 3=UINT32 4=INT32 5=INT64 6=UINT64 7=FP16 | tensor-data-type=0 | | tensor-name | Network input layer name | String | tensor-name=input_1:0 | | scaling-pool-memory-type | Memory type for scaling buffer pool | Integer 0=NVBUF_MEM_DEFAULT 1=NVBUF_MEM_CUDA_PINNED 2=NVBUF_MEM_CUDA_DEVICE 3=NVBUF_MEM_CUDA_UNIFIED 4=NVBUF_MEM_SURFACE_ARRAY | scaling-pool-memory-type=2 | | scaling-pool-compute-hw | Type of computing hardware for scaling | Integer 0=NvBufSurfTransformCompute_Default 1=NvBufSurfTransformCompute_GPU 2=NvBufSurfTransformCompute_VIC | scaling-pool-compute-hw=0 | | scaling-filter | Scaling interpolation method | Integer 0=NvBufSurfTransformInter_Nearest 1=NvBufSurfTransformInter_Bilinear 2=NvBufSurfTransformInter_Algo 3=NvBufSurfTransformInter_Algo2 4=NvBufSurfTransformInter_Algo3 5=NvBufSurfTransformInter_Algo4 6=NvBufSurfTransformInter_Default | scaling-filter=0 | | custom-lib-path | Path of custom library .so file | String | custom-lib-path=/opt/nvidia/deepstream/deepstream/lib/gst-plugins/libcustom2d_preprocess.so | | custom-tensor-preparation-function | Name of tensor preparation function from custom lib | String | custom-tensor-preparation-function=CustomTensorPreparation | **`Gst-nvdspreprocess group- Group Supported Keys`** | Property | Meaning | Type and Range | Example | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------- | --------------------------------------------------------- | | src-ids | Source IDs on which this group applies | Semicolon delimited integer array of values >=0 | src-ids=0;1;2;3 | | custom-input-transformation-function | Name of Custom input transformation function from custom library | String | custom-input-transformation-function=CustomTransformation | | process-on-roi | If enabled process on ROIs else on full-frames (ignored if process-on-frame disabled) | Boolean | process-on-roi=1 | | roi-params-src-\ | Roi coordinates for source \. for each ROI specify left;top;width;height defining the ROI if process-on-roi enabled. | Semicolon delimited integer array of values >=0 | roi-params-0=0;540;900;500;960;0;900;500 | | operate-on-class-ids | Preprocessing is done only on the objects belongs to these class-ids (ignored if process-on-frame enabled) | Semicolon delimited integer(class-ids) array of values >=0 | operate-on-class-ids=0;1 | | process-on-all-objects | If enabled process all the objects else process the objects within the roi (ignored if process-on-frame enabled) | Boolean | process-on-all-objects=1 | | draw-roi | If enabled specified roi's are drawn on display else roi's are not drawn | Boolean | draw-roi=1 | | roi-color | Color of roi to be displayed | Semicolon delimited color components values R;G;B;A | roi_color=0;1;1;1 | | input-object-min-width | Minimum width of objects which are to be processed | Integer, ≥0 | input-object-min-width=100 | | input-object-min-height | Minimum height of objects which are to be processed | Integer, ≥0 | input-object-min-height=100 | | input-object-max-width | Maximum width of objects which are to be processed | Integer, ≥0 | input-object-max-width=500 | | input-object-max-height | Maximum height of objects which are to be processed | Integer, ≥0 | input-object-max-height=500 | The following table describes the keys supported for `[user-configs]` group for custom library `nvdspreprocess_lib`. **Gst-nvdspreprocess user-configs Group Supported Keys** | Property | Meaning | Type and Range | Example | | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- | ----------------------------------------- | | pixel-normalization-factor | factor at which pixels are scaled | Float | pixel-normalization-factor=0.031 | | offsets | Array of mean values of color components to be subtracted from each pixel. Array length must equal the number of color components in the frame. The plugin multiplies mean values by pixel-normalization-factor | Semicolon delimited float array of values ≥0 | offsets=77.5;21.2;11.8 | | mean-file | pathname of mean data file (PPM format) | String | mean-file=/home/ubuntu/model_meanfile.ppm | The following table describes the keys supported for `[user-configs]` group for custom library `libnvds_custom_sequence_preprocess.so`. which gets used in `deepstream-3d-action-recognition` sample app. **user-configs properties for custom sequence preprocess** | Property | Meaning | Type and Range | Example | | --------------------- | ----------------------------------------------------- | ------------------------------- | ---------------------------------------------------------- | | channel-scale-factors | scale factor list for each channel | Semicolon delimited float array | channel-scale-factors= 0.007843137;0.007843137;0.007843137 | | channel-mean-offsets | data mean offsets for each channel | Semicolon delimited float array | channel-mean-offsets=127.5;127.5;127.5 | | stride | sequence sliding stride for each batched sequence | Unsigned Integer, value >= 1 | stride=1 | | subsample | Subsample rates for inference images in each sequence | Unsigned Integer, value >= 0 | subsample=0 | ## Gst Properties The following table describes the Gst properties of the Gst-nvdspreprocess plugin. **Gst-nvspreprocess gst properties** | Property | Meaning | Type and Range | Example notes | | ----------------- | ---------------------------------------------------------------------------------------- | -------------------------------------------------------- | --------------------------------- | | unique-id | Uniquely identify the metadata generated by this element | Integer, \| 0 to 4,294,967,295 | unique-id=1 | | gpu-id | Device ID of GPU to use for pre-processing (dGPU only) | Integer,0-4,294,967,295 | gpu-id=1 | | config-file | Path of configuration file for the Gst-nvdspreprocess element | String | config-file=config_preprocess.txt | | enable | Enable gst-nvdspreprocess plugin or set in passthrough mode | Boolean | enable=1 | | process-on-frame | Preprocessing Modes 1=PGIE Mode 0=SGIE Mode | Boolean | process-on-frame=1 | | target-unique-ids | List of component gie-id for which tensor is prepared | Semicolon delimited integer(gie-ids) array of values >=0 | target-unique-ids=3;4;5 | | operate-on-gie-id | Unique gie-id whose metadata is to be preprocessed (ignored if process-on-frame enabled) | Integer, 0 to 4,294,967,295 | operate-on-gie-id=1 | ## Sample pipelines Given below are some sample pipelines to demonstrate preprocess plugin’s usage, please set the appropriate configuration file and library paths. **Preprocess in PGIE mode for single stream (dGPU):** ``` gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! \ nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt \ input-tensor-meta=1 batch-size=7 ! nvmultistreamtiler width=1920 height=1080 ! nvvideoconvert ! nvdsosd ! nveglglessink ``` **Preprocess in PGIE mode for single stream (Jetson):** ``` gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! \ nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt \ input-tensor-meta=1 batch-size=2 ! nvmultistreamtiler width=1920 height=1080 ! nvvideoconvert ! nvdsosd ! nv3dsink ``` **Preprocess in PGIE mode for Multi-stream (dGPU):** ``` gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=4 width=1920 height=1080 ! nvvideoconvert ! \ nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt \ input-tensor-meta=1 batch-size=8 ! nvmultistreamtiler width=1920 height=1080 ! nvvideoconvert ! nvdsosd ! nveglglessink \ filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! \ m.sink_1 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! \ nvv4l2decoder ! m.sink_2 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_3 ``` **Preprocess in PGIE mode for Multi-stream (Jetson):** ``` gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=4 width=1920 height=1080 ! nvvideoconvert ! \ nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt \ input-tensor-meta=1 batch-size=8 ! nvmultistreamtiler width=1920 height=1080 ! nvvideoconvert ! nvdsosd ! nv3dsink \ filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! \ m.sink_1 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! \ nvv4l2decoder ! m.sink_2 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_3 ``` **Preprocess in PGIE and SGIE Mode for single stream (dGPU):** ``` gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! \ nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt unique-id=1 \ batch-size=2 input-tensor-meta=1 ! nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess_sgie.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt input-tensor-meta=1 unique-id=3 ! \ nvmultistreamtiler width=1920 height=1080 ! nvvideoconvert ! nvdsosd ! nveglglessink ``` **Preprocess in PGIE and SGIE Mode for single stream (Jetson):** ``` gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! \ nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt unique-id=1 \ batch-size=2 input-tensor-meta=1 ! nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess_sgie.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt input-tensor-meta=1 unique-id=3 ! \ nvmultistreamtiler width=1920 height=1080 ! nvvideoconvert ! nvdsosd ! nv3dsink ``` **Preprocess in PGIE and SGIE Mode for Multi-stream (dGPU):** ``` gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! \ nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt unique-id=1 \ batch-size=8 input-tensor-meta=1 ! nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess_sgie.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt input-tensor-meta=1 unique-id=3 ! \ nvmultistreamtiler width=1920 height=1080 ! nvvideoconvert ! nvdsosd ! nveglglessink filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! \ qtdemux ! h264parse ! nvv4l2decoder ! m.sink_1 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! \ m.sink_2 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_3 ``` **Preprocess in PGIE and SGIE Mode for Multi-stream (Jetson):** ``` gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! \ h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! \ nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt unique-id=1 \ batch-size=8 input-tensor-meta=1 ! nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess_sgie.txt ! \ nvinfer config-file-path= /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt input-tensor-meta=1 unique-id=3 ! \ nvmultistreamtiler width=1920 height=1080 ! nvvideoconvert ! nvdsosd ! nv3dsink filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! \ qtdemux ! h264parse ! nvv4l2decoder ! m.sink_1 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! \ m.sink_2 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_3 ```