Application Customization

The NVIDIA® DeepStream SDK on NVIDIA® Tesla® or NVIDIA® Jetson platforms can be customized to support custom neural networks for object detection and classification.
You can create your own model. You must specify the applicable configuration parameters in the [property] group of the nvinfer configuration file (for example, config_infer_primary.txt).
The configuration parameters that you must specify include:
model-file (Caffe model)
proto-file (Caffe model)
uff-file (UFF models)
onnx-file (ONNX models)
model-engine-file, if already generated
int8-calib-file for INT8 mode
mean-file, if required
offsets, if required
maintain-aspect-ratio, if required
parse-bbox-func-name (detectors only)
parse-classifier-func-name (classifiers only)
output-blob-names (Caffe and UFF models)

Custom Model Implementation Interface

nvinfer supports interfaces for these purposes:
Custom bounding box parsing for custom neural network detectors and classifiers
IPlugin implementation for layers not natively supported by NVIDIA® TensorRT™
Initializing non-image input layers in cases where the network has more than one input layer
Creating a CUDA engine using TensorRT Layer APIs instead of model parsing APIs
IModelParser interface to parse the model and fill the layers in an INetworkDefintion
All the interface implementations for the models must go into a single independent shared library. nvinfer dynamically loads the library with dlopen(), looks for implemented interfaces with dlsym(), and calls the interfaces as required.
For more information about the interface, refer to the header file nvdsinfer_custom_impl.h.

Custom Output Parsing

For detectors, you must write a library that can parse the bounding box coordinates and the object class from the output layers. For classifiers, the library must parse the object attributes from the output layers. You can find example code and makefile in the source directory in sources/libs/nvdsinfer_customparser.
The generated library path and the function name must be specified with the configuration parameters as mentioned in the section Custom Model. The README file in sources/libs/nvdsinfer_customparser has an example of how to use this custom parser.

IPlugin Implementation

DeepStream supports networks containing layers not supported by TensorRT but supported through implementations of the IPlugin interface. The objectDetector_SSD, objectDetector_FasterRCNN, and objectDetector_YoloV3 sample applications show examples of IPlugin implementations.

Input Layer Initialization

DeepStream supports initializing non-image input layers for networks having more than one input layer. The layers are initialized only once before the first inference call. The objectDetector_FasterRCNN sample application shows an example of an implementation.

CUDA Engine Creation for Custom Models

DeepStream supports creating TensorRT CUDA engines for models which are not in Caffe, UFF, or ONNX format, or which must be created from TensorRT Layer APIs. The objectDetector_YoloV3 sample application shows an example of the implementation. When using a single custom library for multiple nvinfer plugin instances in a pipeline, each instance can have its own implementation of engine-create-func-name and this can be specified in the configuration file. An example would be back-to-back detector pipeline with different types of yolo models.

IModelParser Interface for Custom Model Parsing

This is an alternative to the “CUDA Engine Creation” interface for parsing and filling a TensorRT network (INetworkDefinition). The objectDetector_YoloV3 sample application shows an example of the implementation.