 |
NVIDIA DeepStream SDK API Reference4.0.2 Release |
|
|
NVIDIA DeepStream SDK API Reference4.0.2 Release |
Copyright (c) 2017-2019, NVIDIA CORPORATION.
All rights reserved.
NVIDIA Corporation and its licensors retain all intellectual property and proprietary rights in and to this software, related documentation and any modifications thereto. Any use, reproduction, disclosure or distribution of this software and related documentation without an express license agreement from NVIDIA Corporation is strictly prohibited.
NVIDIA DeepStream Specification for Custom Method Implementations for custom models
Description: This file specifies the APIs and function definitions to implement custom methods required by DeepStream nvinfer GStreamer plugin to infer using custom models.
All the custom functionality must be implemented in an independent shared library. This library will be dynamically loaded (dlopen()) by "nvinfer" and implemented custom methods will be called as required. The custom library can be specified in the nvinfer element configuration file through the custom-lib-name property.
This section describes the custom bounding box parsing function for custom detector models.
The custom parsing function should be of the type NvDsInferParseCustomFunc. The custom parsing function can be specified in the nvinfer element configuration file using the property parse-bbox-func-name (Name of the parsing function) in addition to custom-lib-name and setting parse-func to 0 (custom).
The nvinfer plugin loads the library and looks for the custom parsing function symbol. The function is called after each inference call is executed.
The macro CHECK_CUSTOM_PARSE_FUNC_PROTOTYPE() can be called after the function definition to validate the function definition.
Based on the type of the model (Caffe or UFF), the library must implement one of the two functions NvDsInferPluginFactoryCaffeGet or NvDsInferPluginFactoryUffGet. During model parsing, the DeepStream "nvinfer" plugin will look for one of the two symbols in the custom library based on the model framework. If the symbol is found, the plugin will call the "Get" function to get a pointer to PluginFactory instance required for parsing.
If the IPluginFactory needs to be used during deserialization of cuda engines, the library must implement NvDsInferPluginFactoryRuntimeGet.
All the three Get functions have a corresponding Destroy functions which will be called if defined when the returned PluginFactory need to be destroyed.
Libraries implementing this interface must use the same function names as in the header file. The "nvinfer" plugin will dynamically load the library and look for the same symbol names.
Refer the FasterRCNN sample provided with the SDK for a sample implementation of the interface.
By default "nvinfer" works with networks having only one input layer for video frames. If the network has more than one input layer, the custom library can implement NvDsInferInitializeInputLayers interface for initializing the other input layers. "nvinfer" assumes that the other input layers have static input information and hence this method is called only once before the first inference.
Refer the FasterRCNN sample provided with the SDK for a sample implementation of the interface.
The NvDsInferCudaEngineGet interface can be used to create and build custom networks not directly supported by nvinfer.
The "nvinfer" plugin will dynamically load the custom library and look for the "NvDsInferCudaEngineGet" symbol.
The interface implementation shall build and return the CudaEngine interface using the supplied nvinfer1::IBuilder instance. The builder instance will already be configured with properties like MaxBatchSize, MaxWorkspaceSize, INT8/FP16 precision parameters etc. The builder instance is managed by "nvinfer" and should not be destroyed by the implementation.
The path to the config file for nvinfer instance is supplied to the interface. Custom properties required by the model can be added to the config file and parsed in the interface implementation.
Definition in file nvdsinfer_custom_impl.h.
Go to the source code of this file.
Data Structures | |
| struct | NvDsInferParseDetectionParams |
| Holds the detection parameters required for parsing objects. More... | |
| union | NvDsInferPluginFactoryCaffe |
| Holds the pointer to a heap allocated Plugin Factory object required during Caffemodel parsing. More... | |
| union | NvDsInferPluginFactoryUff |
| Holds the pointer to a heap allocated Plugin Factory object required during UFF model parsing. More... | |
Macros | |
| #define | CHECK_CUSTOM_PARSE_FUNC_PROTOTYPE(customParseFunc) |
| Macro to validate the custom parser function definition. More... | |
| #define | CHECK_CUSTOM_CLASSIFIER_PARSE_FUNC_PROTOTYPE(customParseFunc) |
| Macro to validate the classifier custom parser function definition. More... | |
Typedefs | |
| typedef bool(* | NvDsInferParseCustomFunc )(std::vector< NvDsInferLayerInfo > const &outputLayersInfo, NvDsInferNetworkInfo const &networkInfo, NvDsInferParseDetectionParams const &detectionParams, std::vector< NvDsInferObjectDetectionInfo > &objectList) |
| Function definition for the custom bounding box parsing function. More... | |
| typedef bool(* | NvDsInferClassiferParseCustomFunc )(std::vector< NvDsInferLayerInfo > const &outputLayersInfo, NvDsInferNetworkInfo const &networkInfo, float classifierThreshold, std::vector< NvDsInferAttribute > &attrList, std::string &descString) |
| Function definition for the custom classifier output parsing function. More... | |
| typedef struct _NvDsInferContextInitParams | NvDsInferContextInitParams |
Enumerations | |
| enum | NvDsInferPluginFactoryType { PLUGIN_FACTORY, PLUGIN_FACTORY_EXT, PLUGIN_FACTORY_V2 } |
| Specifies the type of the Plugin Factory. More... | |
Functions | |
| bool | NvDsInferPluginFactoryCaffeGet (NvDsInferPluginFactoryCaffe &pluginFactory, NvDsInferPluginFactoryType &type) |
| Returns an instance of a newly allocated Plugin Factory interface to be used during parsing of caffe models. More... | |
| void | NvDsInferPluginFactoryCaffeDestroy (NvDsInferPluginFactoryCaffe &pluginFactory) |
Destroy the Plugin Factory instance returned in NvDsInferPluginFactoryCaffeGet More... | |
| bool | NvDsInferPluginFactoryUffGet (NvDsInferPluginFactoryUff &pluginFactory, NvDsInferPluginFactoryType &type) |
| Returns an instance of a newly allocated Plugin Factory interface to be used during parsing of UFF models. More... | |
| void | NvDsInferPluginFactoryUffDestroy (NvDsInferPluginFactoryUff &pluginFactory) |
Destroy the Plugin Factory instance returned in NvDsInferPluginFactoryUffGet More... | |
| bool | NvDsInferPluginFactoryRuntimeGet (nvinfer1::IPluginFactory *&pluginFactory) |
| Returns an instance of a newly allocated Plugin Factory interface to be used during parsing deserialization of cuda engines. More... | |
| void | NvDsInferPluginFactoryRuntimeDestroy (nvinfer1::IPluginFactory *pluginFactory) |
Destroy the PluginFactory instance returned in NvDsInferPluginFactoryRuntimeGet More... | |
| bool | NvDsInferInitializeInputLayers (std::vector< NvDsInferLayerInfo > const &inputLayersInfo, NvDsInferNetworkInfo const &networkInfo, unsigned int maxBatchSize) |
| Initialize the input layers for inference. More... | |
| bool | NvDsInferCudaEngineGet (nvinfer1::IBuilder *builder, NvDsInferContextInitParams *initParams, nvinfer1::DataType dataType, nvinfer1::ICudaEngine *&cudaEngine) |
| Build and return CudaEngine for custom models. More... | |
| #define CHECK_CUSTOM_CLASSIFIER_PARSE_FUNC_PROTOTYPE | ( | customParseFunc | ) |
Macro to validate the classifier custom parser function definition.
Should be called after defining the function.
Definition at line 181 of file nvdsinfer_custom_impl.h.
| #define CHECK_CUSTOM_PARSE_FUNC_PROTOTYPE | ( | customParseFunc | ) |
Macro to validate the custom parser function definition.
Should be called after defining the function.
Definition at line 150 of file nvdsinfer_custom_impl.h.
| typedef bool(* NvDsInferClassiferParseCustomFunc)(std::vector< NvDsInferLayerInfo > const &outputLayersInfo, NvDsInferNetworkInfo const &networkInfo, float classifierThreshold, std::vector< NvDsInferAttribute > &attrList, std::string &descString) |
Function definition for the custom classifier output parsing function.
| [in] | outputLayersInfo | Vector containing information on the output layers of the model. |
| [in] | networkInfo | Network information. |
| [in] | classifierThreshold | Classification confidence threshold |
| [out] | attrList | Reference to a vector in which the function should add the parsed attributes. |
| [out] | descString | Reference to a string object in which the function may place a description string. |
Definition at line 170 of file nvdsinfer_custom_impl.h.
| typedef struct _NvDsInferContextInitParams NvDsInferContextInitParams |
Definition at line 314 of file nvdsinfer_custom_impl.h.
| typedef bool(* NvDsInferParseCustomFunc)(std::vector< NvDsInferLayerInfo > const &outputLayersInfo, NvDsInferNetworkInfo const &networkInfo, NvDsInferParseDetectionParams const &detectionParams, std::vector< NvDsInferObjectDetectionInfo > &objectList) |
Function definition for the custom bounding box parsing function.
| [in] | outputLayersInfo | Vector containing information on the output layers of the model. |
| [in] | networkInfo | Network information. |
| [in] | detectionParams | Detection parameters required for parsing objects. |
| [out] | objectList | Reference to a vector in which the function should add the parsed objects. |
Definition at line 140 of file nvdsinfer_custom_impl.h.
Specifies the type of the Plugin Factory.
Definition at line 193 of file nvdsinfer_custom_impl.h.
| bool NvDsInferCudaEngineGet | ( | nvinfer1::IBuilder * | builder, |
| NvDsInferContextInitParams * | initParams, | ||
| nvinfer1::DataType | dataType, | ||
| nvinfer1::ICudaEngine *& | cudaEngine | ||
| ) |
Build and return CudaEngine for custom models.
The implementation of this interface should build the nvinfer1::ICudaEngine instance using supplied nvinfer1::IBuilder instance. The builder instance is managed by the caller. The implementation should not destroy the instance.
Properties like MaxBatchSize, MaxWorkspaceSize, INT8/FP16 precision parameters, DLA parameters if applicable will already be set on the builder before being passed to the interface. The corresponding Get functions of nvinfer1::IBuilder interface can be used to get the property values.
The implementation should make sure not to reduce the MaxBatchSize of the returned CudaEngine.
| [in] | builder | nvinfer1::IBuilder instance |
| [in] | initParams | Pointer to the structure used for initializing the NvDsInferContext instance. |
| [in] | dataType | Data precision |
| [out] | cudaEngine | Pointer to the built nvinfer1::ICudaEngine instance. |
| bool NvDsInferInitializeInputLayers | ( | std::vector< NvDsInferLayerInfo > const & | inputLayersInfo, |
| NvDsInferNetworkInfo const & | networkInfo, | ||
| unsigned int | maxBatchSize | ||
| ) |
Initialize the input layers for inference.
This function is only called once during before the first inference call.
| [in] | inputLayersInfo | Vector containing information on the input layers of the model. This does not contain the NvDsInferLayerInfo structure for the layer for video frame input. |
| [in] | networkInfo | Network information. |
| [in] | maxBatchSize | Maximum batch size for inference. The buffers for input layers are allocated for this batch size. |
| void NvDsInferPluginFactoryCaffeDestroy | ( | NvDsInferPluginFactoryCaffe & | pluginFactory | ) |
Destroy the Plugin Factory instance returned in NvDsInferPluginFactoryCaffeGet
| [in] | pluginFactory | Reference to NvDsInferPluginFactoryCaffe union holding the pointer to the Plugin Factory instance returned in NvDsInferPluginFactoryCaffeGet. |
| bool NvDsInferPluginFactoryCaffeGet | ( | NvDsInferPluginFactoryCaffe & | pluginFactory, |
| NvDsInferPluginFactoryType & | type | ||
| ) |
Returns an instance of a newly allocated Plugin Factory interface to be used during parsing of caffe models.
The function must set the correct type and the correct field inside the NvDsInferPluginFactoryCaffe union based on the type of the Plugin Factory (i.e. one of pluginFactory, pluginFactoryExt or pluginFactoryV2).
| [out] | pluginFactory | A reference to the NvDsInferPluginFactoryCaffe union |
| [out] | type | Enum indicating the type of the returned Plugin Factory instance (i.e. the valid field in the NvDsInferPluginFactoryCaffe union) |
| void NvDsInferPluginFactoryRuntimeDestroy | ( | nvinfer1::IPluginFactory * | pluginFactory | ) |
Destroy the PluginFactory instance returned in NvDsInferPluginFactoryRuntimeGet
| [in] | pluginFactory | PluginFactory instance returned in NvDsInferPluginFactoryRuntimeGet |
| bool NvDsInferPluginFactoryRuntimeGet | ( | nvinfer1::IPluginFactory *& | pluginFactory | ) |
Returns an instance of a newly allocated Plugin Factory interface to be used during parsing deserialization of cuda engines.
| [out] | pluginFactory | A reference to nvinfer1::IPluginFactory* in which the function must place the pointer to the instance. |
| void NvDsInferPluginFactoryUffDestroy | ( | NvDsInferPluginFactoryUff & | pluginFactory | ) |
Destroy the Plugin Factory instance returned in NvDsInferPluginFactoryUffGet
| [in] | pluginFactory | Reference to NvDsInferPluginFactoryUff union holding the pointer to the Plugin Factory instance returned in NvDsInferPluginFactoryUffGet. |
| bool NvDsInferPluginFactoryUffGet | ( | NvDsInferPluginFactoryUff & | pluginFactory, |
| NvDsInferPluginFactoryType & | type | ||
| ) |
Returns an instance of a newly allocated Plugin Factory interface to be used during parsing of UFF models.
The function must set the correct type and the correct field inside the NvDsInferPluginFactoryUff union based on the type of the Plugin Factory (i.e. one of pluginFactory or pluginFactoryExt)
| [out] | pluginFactory | A reference to the NvDsInferPluginFactoryUff union |
| [out] | type | Enum indicating the type of the returned Plugin Factory instance (i.e. the valid field in the NvDsInferPluginFactoryUff union) |
