> This page is for version 9.2 (default). > For other versions, use one of these documentation indexes: > - 9.2 (default): https://docs.nvidia.com/deepstream/9.2/llms.txt > - Archives: https://docs.nvidia.com/deepstream/archives/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/deepstream/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/deepstream/_mcp/server. # Gst-nvdsxfer The `Gst-nvdsxfer` plugin performs data transfer between discrete GPUs. Currently it is supported on x86 platform only. It uses CUDA APIs to utilize NVIDIA NVLINK technology for high-speed, direct GPU-to-GPU interconnect for optimized data transfer between discrete GPUs. The plugin accepts CUDA memory based (`NvBufSurface` allocated) video `Gst` Buffers from upstream component. It transfer the input data to CUDA memory based (`NvBufSurface` allocated) video output `Gst` Buffer using the optimized NVLINK based data copy. > **Note** > > The `Gst-nvdsxfer` plugin is currently supports Single Node, Single Application with Multi-dGPU setup based use case pipelines. Video format conversion or scaling is not supported while doing data copy between two discrete GPUs. Multi-dGPUs are connected using NVLINK Bridge Connector. Users must confirm the NVLINK state (active/inactive - Use command "`nvidia-smi nvlink -s`" to check) between two discrete GPUs before using the `nvdsxfer` plugin in the `gst-pipeline`. As shown in the diagram below input video data is copied to output over a NVLINK connected discrete GPUs. ![Gst-Nvdsxfer](/deepstream/_fern-img/2e5013e4f1261eea82e24634f1d6e347c6e2d01095cae0af9aa1186ee0298a8e.webp) ## Inputs and Outputs This section summarizes the inputs, outputs of the `Gst-nvdsxfer` plugin. * Inputs * Gst Buffer batched buffer * NvDsBatchMeta * Raw Video Format: NV12, I420, RGBA (NVMM) * Control parameters * gpu-id * p2p-gpu-id * batch-size * buffer-pool-size * nvbuf-memory-type * Output * Gst Buffer batched buffer * NvDsBatchMeta * Raw Video Format: NV12, I420, RGBA (NVMM) ## Gst Properties The following tables describes the `Gst` properties of the `Gst-nvdsxfer` plugin. **nvdsxfer plugin properties** | Property | Meaning | Type and Range | Example notes | | ----------------- | -------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | | batch-size | Maximum number of buffers in a batch | Unsigned Integer. Range: 0 - 4294967295 Default: 1 | batch-size=2 | | buffer-pool-size | Maximum number of buffers in muxer's internal pool | Unsigned Integer. Range: 2 - 16 Default: 4 | buffer-pool-size=2 | | gpu-id | Set GPU Device ID | Unsigned Integer. Range: 0 - 4294967295 Default: 0 | gpu-id=1 | | p2p-gpu-id | Set P2P GPU ID to enable P2P access.Default P2P access between GPUs is disabled. | Integer. Range: -1 - 1024 Default: -1 | p2p-gpu-id=0 | | nvbuf-memory-type | Type of NvBufSurface Memory to be allocated for output buffers | Enum "GstNvBufMemoryType" Default: 0, "nvbuf-mem-default (0): nvbuf-mem-default - Default memory allocated, specific to particular platform (1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory (2): nvbuf-mem-cuda-device - Allocate Device cuda memory (3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory | nvbuf-memory-type=2 | ## How to test a) `nvdsxfer` is currently supported for X86 only. support with "`Jetson + dGPU`" is not yet enabled. b) Multi-dGPUs are connected using NVLINK Bridge Connector. Use below command to confirm the NVLINK state (active/inactive) if ready to use. ``` nvidia-smi nvlink -s ``` c) `nvdsxfer` plugin currently verified using 2 separate dGPU (discrete GPUs) only.Below listed `gst-launch-1.0 pipelines` simulates some of the reference use cases pipelines using 2 separate dGPU (discrete GPUs). d) Set property `p2p_gpu_id=0` if Peer to Peer (P2P) access between discrete GPUs permitted. e) If P2P access is not possible then pipeline will fail, remove `p2p_gpu_id=0` property to make it run without P2P access. f) Below mentioned reference `gst-launch-1.0 pipelines` use legacy `streammux` by default. New `nvstreammux` can also be used by enabling USE_NEW_NVSTREAMMUX=yes environment variable with appropriate properties set for new `streammux` plugin > **Note** > > `gst-launch-1.0 pipelines` mentioned in the [Use cases](#use-cases) section, are not optimal pipelines though can demonstrate `nvdsxfer` plugin usage for various use cases to achieve better performance and GPU utilization. > **Note** > > `deepstream-multigpu-nvlink-test` sample application can demonstrate nvdsxfer plugin based gstreamer pipeline. > Refer `/opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-multigpu-nvlink-test/README` for usage and application supported feature. ## Use cases Below reference `gst-launch-1.0` commands uses h265 elementary stream as input. In order to use DeepStream SDK provided sample mp4 stream, use following command to convert from mp4 container file to h265 elementary stream. ```text cd /opt/nvidia/deepstream/deepstream/ gst-launch-1.0 filesrc location= samples/streams/sample_1080p_h265.mp4 ! qtdemux ! h265parse ! 'video/x-h265,stream-format=byte-stream' ! filesink location= samples/streams/sample_1080p.h265 ``` ### Single Stream + Multi-dGPUs Setup A) Running "Decode + StreamMux + PGIE" and "Tracker + SGIE (Multiple)" on separate dGPUs ![Gst-Nvdsxfer](/deepstream/_fern-img/67019eb907c4208da83778958a14f682fa58621bae87c927c2c32449a6aa825f.webp) ``` gst-launch-1.0 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_0 nvstreammux name=m batch-size=1 gpu-id=0 \ width=1920 height=1080 nvbuf-memory-type=2 ! queue ! nvinfer gpu-id=0 batch-size=1 \ config-file-path=samples/configs/deepstream-app/config_infer_primary.txt ! queue ! \ nvdsxfer gpu-id=1 p2p_gpu_id=0 ! queue ! nvtracker gpu-id=1 enable-batch-process=1 \ ll-lib-file=lib/libnvds_nvmultiobjecttracker.so ll-config-file=samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! \ queue ! nvinfer gpu-id=1 batch-size=16 unique-id=2 config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehiclemake.txt ! \ queue ! nvinfer gpu-id=1 batch-size=16 unique-id=3 config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt ! \ queue ! fpsdisplaysink video-sink=fakesink sync=0 -e -v ``` B) Running "Decode + StreamMux" and "PGIE + Tracker + SGIE (Multiple)" on separate dGPUs ![Gst-Nvdsxfer](/deepstream/_fern-img/8e9471eb46038d4af01db00c6066939f5d78fb9228473106aa23562e49f13a1a.webp) ``` gst-launch-1.0 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_0 nvstreammux name=m batch-size=1 gpu-id=0 \ width=1920 height=1080 nvbuf-memory-type=2 ! queue ! nvdsxfer gpu-id=1 p2p_gpu_id=0 ! queue ! \ nvinfer gpu-id=1 batch-size=1 config-file-path= samples/configs/deepstream-app/config_infer_primary.txt ! queue ! \ nvtracker gpu-id=1 enable-batch-process=1 ll-lib-file=lib/libnvds_nvmultiobjecttracker.so \ ll-config-file=samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! queue ! nvinfer gpu-id=1 batch-size=16 unique-id=2 \ config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehiclemake.txt ! queue ! nvinfer gpu-id=1 batch-size=16 unique-id=3 \ config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt ! queue ! fpsdisplaysink video-sink=fakesink sync=0 -e -v ``` ### Multiple Streams + Multi-dGPU Setup A) Running "Multi-instance(4) decode + Streammux + PGIE" on single dGPU and " tracker + SGIE - multiple models" on separate dGPU ![Gst-Nvdsxfer](/deepstream/_fern-img/6d84b66b954f8b1adbfbfe4beb95faf93e30fd99a9d2ad663e8e49846aedf098.webp) ``` gst-launch-1.0 nvstreammux name=m batch-size=4 gpu-id=0 width=1920 height=1080 nvbuf-memory-type=2 ! queue ! \ nvinfer gpu-id=0 batch-size=4 config-file-path=samples/configs/deepstream-app/config_infer_primary.txt ! queue ! \ nvdsxfer gpu-id=1 p2p_gpu_id=0 ! queue ! nvtracker gpu-id=1 enable-batch-process=1 \ ll-lib-file=lib/libnvds_nvmultiobjecttracker.so ll-config-file=samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! queue ! \ nvinfer gpu-id=1 batch-size=16 unique-id=2 config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehiclemake.txt ! queue ! \ nvinfer gpu-id=1 batch-size=16 unique-id=3 config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt ! queue ! \ fpsdisplaysink video-sink=fakesink sync=0 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_0 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_1 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_2 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_3 -e -v ``` B) Running "Multi-instance(4) decode + StreamMux" on single-dGPU and "PGIE + Tracker + SGIE(Multiple)" on separate dGPU ![Gst-Nvdsxfer](/deepstream/_fern-img/22db2e26ecb2bb6f469ad7bfa23ce0917ed61a288965a7eeaa04d3ce9ffaafbd.webp) ``` gst-launch-1.0 nvstreammux name=m batch-size=4 gpu-id=0 width=1920 height=1080 nvbuf-memory-type=2 ! queue ! \ nvdsxfer gpu-id=1 p2p_gpu_id=0 ! queue ! nvinfer gpu-id=1 batch-size=4 config-file-path=samples/configs/deepstream-app/config_infer_primary.txt ! queue ! \ nvtracker gpu-id=1 enable-batch-process=1 ll-lib-file=lib/libnvds_nvmultiobjecttracker.so \ ll-config-file=samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! queue ! nvinfer gpu-id=1 batch-size=16 unique-id=2 \ config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehiclemake.txt ! queue ! nvinfer gpu-id=1 batch-size=16 unique-id=3 \ config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt ! queue ! fpsdisplaysink video-sink=fakesink sync=0 \ multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_0 \ multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_1 \ multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_2 \ multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_3 -e -v ``` C) Running "Multi-instance(8) decode" on multi-dGPU and "StreamMux + PGIE + Tracker + SGIE(Multiple)" on any one of the dGPU ![Gst-Nvdsxfer](/deepstream/_fern-img/07e08e6c3f98588c3349391895a9da54745c3c7e705f134c6f036696ed5df3ea.webp) ``` gst-launch-1.0 nvstreammux name=m batch-size=8 gpu-id=0 width=1920 height=1080 nvbuf-memory-type=2 ! queue ! nvinfer gpu-id=0 batch-size=8 \ config-file-path=samples/configs/deepstream-app/config_infer_primary.txt ! queue ! nvtracker gpu-id=0 enable-batch-process=1 \ ll-lib-file=lib/libnvds_nvmultiobjecttracker.so ll-config-file=samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! queue ! \ nvinfer gpu-id=0 batch-size=16 unique-id=2 config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehiclemake.txt ! queue ! \ nvinfer gpu-id=0 batch-size=16 unique-id=3 config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt ! queue ! \ fpsdisplaysink video-sink=fakesink sync=0 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_0 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_1 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_2 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_3 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=1 cudadec-memtype=0 ! queue ! nvdsxfer gpu-id=0 p2p_gpu_id=1 ! queue ! m.sink_4 multifilesrc \ location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! nvv4l2decoder gpu-id=1 cudadec-memtype=0 ! queue ! \ nvdsxfer gpu-id=0 p2p_gpu_id=1 ! queue ! m.sink_5 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=1 cudadec-memtype=0 ! queue ! nvdsxfer gpu-id=0 p2p_gpu_id=1 ! queue ! m.sink_6 multifilesrc \ location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! nvv4l2decoder gpu-id=1 cudadec-memtype=0 ! queue ! \ nvdsxfer gpu-id=0 p2p_gpu_id=1 ! queue ! m.sink_7 -e -v ``` D) Running "Multi-instance(8) decode" on multi-dGPU, "StreamMux + PGIE" and "Tracker + SGIE(Multiple)" on separate dGPU ![Gst-Nvdsxfer](/deepstream/_fern-img/75ea638b76d886a7fbacfb3a8c84bd2d78899ff82b6795552c75431d15d844f8.webp) ``` gst-launch-1.0 nvstreammux name=m batch-size=8 gpu-id=0 width=1920 height=1080 nvbuf-memory-type=2 ! queue ! nvinfer gpu-id=0 batch-size=8 \ config-file-path=samples/configs/deepstream-app/config_infer_primary.txt ! queue ! nvdsxfer gpu-id=1 p2p_gpu_id=0 ! queue ! nvtracker gpu-id=1 \ enable-batch-process=1 ll-lib-file=lib/libnvds_nvmultiobjecttracker.so ll-config-file=samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! \ queue ! nvinfer gpu-id=1 batch-size=16 unique-id=2 config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehiclemake.txt ! \ queue ! nvinfer gpu-id=1 batch-size=16 unique-id=3 config-file-path=samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt ! \ queue ! fpsdisplaysink video-sink=fakesink sync=0 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! \ queue ! nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_0 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! \ queue ! nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_1 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! \ queue ! nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_2 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! \ queue ! nvv4l2decoder gpu-id=0 cudadec-memtype=0 ! queue ! m.sink_3 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! \ queue ! nvv4l2decoder gpu-id=1 cudadec-memtype=0 ! queue ! nvdsxfer gpu-id=0 p2p_gpu_id=1 ! queue ! m.sink_4 multifilesrc \ location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! nvv4l2decoder gpu-id=1 cudadec-memtype=0 ! queue ! \ nvdsxfer gpu-id=0 p2p_gpu_id=1 ! queue ! m.sink_5 multifilesrc location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! \ nvv4l2decoder gpu-id=1 cudadec-memtype=0 ! queue ! nvdsxfer gpu-id=0 p2p_gpu_id=1 ! queue ! m.sink_6 multifilesrc \ location=samples/streams/sample_1080p.h265 loop=true ! h265parse ! queue ! nvv4l2decoder gpu-id=1 cudadec-memtype=0 ! queue ! \ nvdsxfer gpu-id=0 p2p_gpu_id=1 ! queue ! m.sink_7 -e -v ```