The NVIDIA® DeepStream SDK on NVIDIA® Tesla® or NVIDIA® Jetson platforms can be customized to support custom neural networks for object detection and classification.
You can create your own model. You must specify the applicable configuration parameters in the [property] group of the nvinfer configuration file (for example, config_infer_primary.txt).
The configuration parameters that you must specify include:
• model-file (Caffe model)
• proto-file (Caffe model)
• uff-file (UFF models)
• onnx-file (ONNX models)
• model-engine-file, if already generated
• int8-calib-file for INT8 mode
• mean-file, if required
• offsets, if required
• maintain-aspect-ratio, if required
• parse-bbox-func-name (detectors only)
• parse-classifier-func-name (classifiers only)
• custom-lib-path
• output-blob-names (Caffe and UFF models)
• network-type
• model-color-format
• process-mode
Custom Model Implementation Interface
nvinfer supports interfaces for these purposes:
• Custom bounding box parsing for custom neural network detectors
• IPlugin implementation for layers not natively supported by NVIDIA® TensorRT™
• Initializing non-image input layers in cases where the network has more than one input layer
• Creating a CUDA engine using TensorRT Layer APIs instead of model parsing APIs
All of the interface implementations for the models must go into a single independent shared library. nvinfer dynamically loads the library with dlopen(), looks for implemented interfaces with dlsym(), and calls the interfaces as required.
For more information about the interface, refer to the header file nvdsinfer_custom_impl.h.
Custom Output Parsing
For detectors, you must write a library that can parse the bounding box coordinates and the objuect class from the output layers. For classifiers, the library must parse the object attributes from the output layers. You can find example code and makefile in the source directory in sources/libs/nvdsinfer_customparser.
The generated library path and the function name must be specified with the configuration parameters as mentioned in the section Custom Model. The README file in sources/libs/nvdsinfer_customparser has an example of how to use this custom parser.
IPlugin Implementation
DeepStream supports networks containing layers not supported by TensorRT but supported through implementations of the IPlugin interface. The objectDetector_SSD, objectDetector_FasterRCNN, and objectDetector_YoloV3 sample applications show examples of IPlugin implementations.
Input Layer Initialization
DeepStream supports initializing non-image input layers for networks having more than one input layer. The layers are initialized only once before the first inference call. The objectDetector_FasterRCNN sample application shows an example of an implementation.
CUDA Engine Creation for Custom Models
DeepStream supports creating TensorRT CUDA engines for models which are not in Caffe, UFF, or ONNX format, or which must be created from TensorRT Layer APIs. The objectDetector_YoloV3 sample application shows an example of an implementation.
