DeepStream 4.0.1 supports NVIDIA® TensorRT™ plugins for custom layers. The Gst-nvinfer plugin now has support for the IPluginV2 and IPluginCreator interface, introduced in TensorRT 5.0. For caffemodels and for backward compatibility with existing plugins, it also supports the following interfaces:

See the TensorRT documentation for details on new and deprecated plugin interfaces.

To use the new IPluginCreator interface you must implement the interface in an independent custom library. This library must be passed to the Gst-nvinfer plugin through its configuration file by specifying the library’s pathname with the custom-lib-path key.

Gst-nvinfer opens the library with dlopen(), which causes the plugin to be registered with TensorRT. There is no further direct interaction between the custom library and Gst-nvinfer. TensorRT calls the custom plugin functions as required.

The SSD sample provided with the SDK provides an example of using the IPluginV2 and IPluginCreator interface. This sample has been adapted from TensorRT.

To use the IPluginFactory interface, you must implement the interface in an independent custom library. Pass this library to the Gst-nvinfer plugin through the plugin’s configuration file by specifying the library’s pathname in the custom-lib-path key. The custom library must implement the applicable functions:

These structures are defined in nvdsinfer_custom_impl.h. The function definitions must be named as in the header file. Gst-nvinfer opens the custom library with dlopen() and looks for the names.

During parsing and building of a caffe network, Gst-nvinfer looks for NvDsInferPluginFactoryCaffeGet. If found, it calls the function to get the IPluginFactory instance. Depending on the type of IPluginFactory returned, Gst-nvinfer sets the factory using one of the ICaffeParser interface’s methods setPluginFactory(), setPluginFactoryExt(), or setPluginFactoryV2().

After the network has been built and serialized, Gst-nvinfer looks for NvDsInferPluginFactoryCaffeDestroy and calls it to destroy the IPluginFactory instance.

During parsing and building of a caffe network, Gst-nvinfer looks for NvDsInferPluginFactoryUffGet. If found, it calls the function to get the IPluginFactory instance. Depending on the type of IPluginFactory returned, Gst-nvinfer sets the factory using one of the IUffParser inteface’s methods setPluginFactory() or setPluginFactoryExt().

After the network has been built and serialized, Gst-nvinfer looks for NvDsInferPluginFactoryUffDestroy and calls it to destroy the IPluginFactory instance.

If deserializing the models requires an instance of NvInfer1::IPluginFactory, the custom library must also implement NvDsInferPluginFactoryRuntimeGet() and optionally NvDsInferPluginFactoryRuntimeDestroy(). During deserialization, Gst-nvinfer calls the library’s NvDsInferPluginFactoryRuntimeGet() function to get the IPluginFactory instance, then calls NvDsInferPluginFactoryRuntimeDestroy to destroy the instance if it finds that function during Gst-nvinfer deinitialization.

The FasterRCNN sample provided with the SDK provides an example of using the IPluginV2+nvcaffeparser1::IPluginFactoryV2 interface with DeepStream. This sample has been adapted from TensorRT. It also provides an example of using the legacy IPlugin + nvcaffeparser1::IPluginFactory + Gst-nvinfer 1::IPluginFactory interface for backward compatibility.