NVIDIA DeepStream SDK API Reference

4.0.2 Release
 All Data Structures Files Functions Variables Typedefs Enumerations Enumerator Macros Groups Pages
Data Structures
nvdsinfer_context.h File Reference

Detailed Description

Copyright (c) 2018-2019, NVIDIA CORPORATION.

All rights reserved.

NVIDIA Corporation and its licensors retain all intellectual property and proprietary rights in and to this software, related documentation and any modifications thereto. Any use, reproduction, disclosure or distribution of this software and related documentation without an express license agreement from NVIDIA Corporation is strictly prohibited.

NVIDIA DeepStream Inference Interface

Description: This file specifies the DeepStream inference interface APIs.

The DeepStream inference API "NvDsInfer" provides methods to initialize/ deinitialize the inference engine, pre-process the input frames as required by the network, and parse the output from the raw tensor buffers.

Both C and C++ interfaces are available with the C interface being a simple wrapper over the C++ interface.

An opaque handle (NvDsInferContext) to an instance of the context required by the APIs can be created using factory functions createNvDsInferContext / NvDsInferContext_Create. The two functions accept an instance of NvDsInferContextInitParams structure to initialize the context. Logging callbacks can be set to get detailed information about failures/warnings.

Initialization parameters allow configuration of the network data type, type of the network(Detector/Classifier/Other), pre-processing parameters (mean subtraction and normalization), model related parameters like Caffe/Uff/Onnx model file paths, output layer names etc.

Batches of frames can be queued for inferencing using NvDsInferContext::queueInputBatch / NvDsInferContext_QueueInputBatch. The input frame memories should be accessible by the GPU device configured during initialization. An asynchronous callback function can be provided to return the input buffers back to the caller as soon as the input is consumed.

Inference output can be dequeued using NvDsInferContext::dequeueOutputBatch / NvDsInferContext_DequeueOutputBatch. The order dequeued outputs corresponds to the input queueing order. In case of failure, the output of the batch is lost. The dequeued output must be released back to the context using NvDsInferContext::releaseBatchOutput / NvDsInferContext_ReleaseBatchOutput to free the associated memory and return the output layer buffers for re-use by the context.

Detectors will output an array of detected objects for each frame in the batch. Classifiers will classify entire frames and output an array of attributes for each frame. Segmentation will classify each pixel in the frame. A special network type has been provided (Other) where the output layers will not be parsed. The caller can parse the device/host output layer buffers. This can also be used along with DeepStream GStreamer Inference plugin "nvinfer" to flow the output buffers as metadata.

Other methods to get the parsed labels from a labels file and properties of all the layers bound by the inferece engine have been provided.

The NvDsInfer APIs can be extended using the custom method implementations. Refer to the Custom Method Implementations section for more details.

Definition in file nvdsinfer_context.h.

Go to the source code of this file.

Data Structures

struct  NvDsInferDetectionParams
 Holds the detection and bounding box grouping parameters. More...
 
struct  _NvDsInferContextInitParams
 Holds the initialization parameters required for the NvDsInferContext interface. More...
 
struct  NvDsInferContextBatchInput
 Holds the information about one batch to be inferred. More...
 
struct  NvDsInferObject
 Holds the information about one detected object. More...
 
struct  NvDsInferDetectionOutput
 Holds the information on all objects detected by a detector network in one frame. More...
 
struct  NvDsInferClassificationOutput
 Holds the information on all attributes classifed by a classifier network for one frame. More...
 
struct  NvDsInferSegmentationOutput
 Holds the information parsed from segmentation network output for one frame. More...
 
struct  NvDsInferFrameOutput
 Holds the information inferred by the network on one frame. More...
 
struct  NvDsInferContextBatchOutput
 A batch is an array of frames. More...
 

Functions

NvDsInferContext API C-interface

This section describes the C interface for the NvDsInferContext class.

NvDsInferStatus NvDsInferContext_Create (NvDsInferContextHandle *handle, NvDsInferContextInitParams *initParams, void *userCtx, NvDsInferContextLoggingFunc logFunc)
 Creates a new instance of the NvDsInferContext class with the supplied initialization parameters. More...
 
void NvDsInferContext_Destroy (NvDsInferContextHandle handle)
 Destroy a NvDsInferContext instance and release the associated resources. More...
 
NvDsInferStatus NvDsInferContext_QueueInputBatch (NvDsInferContextHandle handle, NvDsInferContextBatchInput *batchInput)
 Queue a batch of input frames for pre-processing and inferencing. More...
 
NvDsInferStatus NvDsInferContext_DequeueOutputBatch (NvDsInferContextHandle handle, NvDsInferContextBatchOutput *batchOutput)
 Dequeue output for a batch of frames. More...
 
void NvDsInferContext_ReleaseBatchOutput (NvDsInferContextHandle handle, NvDsInferContextBatchOutput *batchOutput)
 Free the memory associated with the batch output and release the set of host buffers back to the context for re-use. More...
 
void NvDsInferContext_GetNetworkInfo (NvDsInferContextHandle handle, NvDsInferNetworkInfo *networkInfo)
 Get the network input information. More...
 
unsigned int NvDsInferContext_GetNumLayersInfo (NvDsInferContextHandle handle)
 Get the number of the bound layers of the inference engine in the NvDsInferContext instance. More...
 
void NvDsInferContext_FillLayersInfo (NvDsInferContextHandle handle, NvDsInferLayerInfo *layersInfo)
 Fill the input vector with information on all the bound layers of the inference engine in the NvDsInferContext instance. More...
 
const char * NvDsInferContext_GetLabel (NvDsInferContextHandle handle, unsigned int id, unsigned int value)
 Get the string label associated with the class_id for detectors and the attribute id and the attribute value for classifiers. More...
 

NvDsInferContext API common types and functions.

This section describes the common types and functions for both the C and C++ interfaces for the NvDsInferContext class.

#define _PATH_MAX   4096
 Maximum length of a file path parameter. More...
 
#define _MAX_CHANNELS   4
 Maximum number of channels supported by the API for image input layers. More...
 
#define _MAX_STR_LENGTH   1024
 Maximum length of string parameters. More...
 
#define NVDSINFER_MAX_BATCH_SIZE   1024
 Maximum batch size to be supported by nvdsinfer. More...
 
#define NVDSINFER_MIN_OUTPUT_BUFFERPOOL_SIZE   2
 Minimum number of sets of output buffers that need to be allocated. More...
 
enum  NvDsInferNetworkMode {
  NvDsInferNetworkMode_FP32,
  NvDsInferNetworkMode_INT8,
  NvDsInferNetworkMode_FP16
}
 Enum for internal data format to be used by the inference engine. More...
 
enum  NvDsInferNetworkType {
  NvDsInferNetworkType_Detector,
  NvDsInferNetworkType_Classifier,
  NvDsInferNetworkType_Segmentation,
  NvDsInferNetworkType_Other = 100
}
 Enum for the type of the network. More...
 
enum  NvDsInferFormat {
  NvDsInferFormat_RGB,
  NvDsInferFormat_BGR,
  NvDsInferFormat_GRAY,
  NvDsInferFormat_RGBA,
  NvDsInferFormat_BGRx,
  NvDsInferFormat_Unknown = 0xFFFFFFFF
}
 Enum for color formats. More...
 
enum  NvDsInferUffInputOrder {
  NvDsInferUffInputOrder_kNCHW,
  NvDsInferUffInputOrder_kNHWC,
  NvDsInferUffInputOrder_kNC
}
 Enum for the UFF input layer order. More...
 
enum  NvDsInferStatus {
  NVDSINFER_SUCCESS = 0,
  NVDSINFER_CONFIG_FAILED,
  NVDSINFER_CUSTOM_LIB_FAILED,
  NVDSINFER_INVALID_PARAMS,
  NVDSINFER_OUTPUT_PARSING_FAILED,
  NVDSINFER_CUDA_ERROR,
  NVDSINFER_TENSORRT_ERROR,
  NVDSINFER_UNKNOWN_ERROR
}
 Enum for the status codes returned by NvDsInferContext. More...
 
enum  NvDsInferLogLevel {
  NVDSINFER_LOG_ERROR,
  NVDSINFER_LOG_WARNING,
  NVDSINFER_LOG_INFO,
  NVDSINFER_LOG_DEBUG
}
 Enum for the log levels of NvDsInferContext. More...
 
typedef struct
_NvDsInferContextInitParams 		NvDsInferContextInitParams
 Holds the initialization parameters required for the NvDsInferContext interface. More...
 
typedef void(* NvDsInferContextReturnInputAsyncFunc )(void *data)
 Callback function type for returning the input client buffers back to the NvDsInferContext client asynchronously. More...
 
typedef struct INvDsInferContext * NvDsInferContextHandle
 An opaque pointer type to be used as an handle for the context instance. More...
 
typedef void(* NvDsInferContextLoggingFunc )(NvDsInferContextHandle handle, unsigned int uniqueID, NvDsInferLogLevel logLevel, const char *funcName, const char *logMessage, void *userCtx)
 Callback function type for logging the NvDsInferContext messages. More...
 
void NvDsInferContext_ResetInitParams (NvDsInferContextInitParams *initParams)
 Reset the members of initialization parameters to default values. More...
 
const char * NvDsInferContext_GetStatusName (NvDsInferStatus status)
 Get the string name for the status. More...
 

Macro Definition Documentation

#define _MAX_CHANNELS   4

Maximum number of channels supported by the API for image input layers.

Definition at line 81 of file nvdsinfer_context.h.

#define _MAX_STR_LENGTH   1024

Maximum length of string parameters.

Definition at line 84 of file nvdsinfer_context.h.

#define _PATH_MAX   4096

Maximum length of a file path parameter.

Definition at line 78 of file nvdsinfer_context.h.

#define NVDSINFER_MAX_BATCH_SIZE   1024

Maximum batch size to be supported by nvdsinfer.

Definition at line 87 of file nvdsinfer_context.h.

#define NVDSINFER_MIN_OUTPUT_BUFFERPOOL_SIZE   2

Minimum number of sets of output buffers that need to be allocated.

Definition at line 90 of file nvdsinfer_context.h.

Typedef Documentation

typedef struct INvDsInferContext* NvDsInferContextHandle

An opaque pointer type to be used as an handle for the context instance.

Definition at line 465 of file nvdsinfer_context.h.

typedef struct _NvDsInferContextInitParams NvDsInferContextInitParams

Holds the initialization parameters required for the NvDsInferContext interface.

typedef void(* NvDsInferContextLoggingFunc)(NvDsInferContextHandle handle, unsigned int uniqueID, NvDsInferLogLevel logLevel, const char *funcName, const char *logMessage, void *userCtx)

Callback function type for logging the NvDsInferContext messages.

Parameters
[in]handleHandle of the NvDsInferContext instance which generated the log.
[in]uniqueIDUniqueID of the NvDsInferContext instance which generated the log.
[in]logLevelLevel of the log.
[in]funcNameName of the function which generated the log.
[in]logMessageThe log message string.
[in]userCtxThe opaque user context pointer supplied when creating the NvDsInferContext instance.

Definition at line 479 of file nvdsinfer_context.h.

typedef void(* NvDsInferContextReturnInputAsyncFunc)(void *data)

Callback function type for returning the input client buffers back to the NvDsInferContext client asynchronously.

Parameters
[in]dataThe opaque pointer provided to input queueing function through NvDsInferContextBatchInput.

Definition at line 332 of file nvdsinfer_context.h.

Enumeration Type Documentation

enum NvDsInferFormat

Enum for color formats.

Enumerator
NvDsInferFormat_RGB 

24-bit interleaved R-G-B

NvDsInferFormat_BGR 

24-bit interleaved B-G-R

NvDsInferFormat_GRAY 

8-bit Luma

NvDsInferFormat_RGBA 

32-bit interleaved R-G-B-A

NvDsInferFormat_BGRx 

32-bit interleaved B-G-R-x

NvDsInferFormat_Unknown 

Definition at line 124 of file nvdsinfer_context.h.

enum NvDsInferLogLevel

Enum for the log levels of NvDsInferContext.

Enumerator
NVDSINFER_LOG_ERROR 
NVDSINFER_LOG_WARNING 
NVDSINFER_LOG_INFO 
NVDSINFER_LOG_DEBUG 

Definition at line 195 of file nvdsinfer_context.h.

enum NvDsInferNetworkMode

Enum for internal data format to be used by the inference engine.

Enumerator
NvDsInferNetworkMode_FP32 
NvDsInferNetworkMode_INT8 
NvDsInferNetworkMode_FP16 

Definition at line 95 of file nvdsinfer_context.h.

enum NvDsInferNetworkType

Enum for the type of the network.

Enumerator
NvDsInferNetworkType_Detector 

Detectors will find objects and their coordinates in the input frame along with the class of the object.

NvDsInferNetworkType_Classifier 

Classifiers - will classify the entire frame into some finite possible classes.

NvDsInferNetworkType_Segmentation 

Segmentation - will classify each pixel into some finite possible classes.

NvDsInferNetworkType_Other 

Other - Output layers will not be parsed by NvDsInferContext.

This is useful for networks producing custom output. The output can be parsed by the NvDsInferContext client or can be combined with "gst-nvinfer" feature to flow output tensors as metadata.

Definition at line 105 of file nvdsinfer_context.h.

enum NvDsInferStatus

Enum for the status codes returned by NvDsInferContext.

Enumerator
NVDSINFER_SUCCESS 

NvDsInferContext operation succeeded.

NVDSINFER_CONFIG_FAILED 

Failed to configure the NvDsInferContext instance possibly due to an erroneous initialization property.

NVDSINFER_CUSTOM_LIB_FAILED 

Custom Library interface implementation failed.

NVDSINFER_INVALID_PARAMS 

Invalid parameters were supplied.

NVDSINFER_OUTPUT_PARSING_FAILED 

Output parsing failed.

NVDSINFER_CUDA_ERROR 

CUDA error was encountered.

NVDSINFER_TENSORRT_ERROR 

TensorRT interface failed.

NVDSINFER_UNKNOWN_ERROR 

Unknown error was encountered.

Definition at line 171 of file nvdsinfer_context.h.

enum NvDsInferUffInputOrder

Enum for the UFF input layer order.

Enumerator
NvDsInferUffInputOrder_kNCHW 
NvDsInferUffInputOrder_kNHWC 
NvDsInferUffInputOrder_kNC 

Definition at line 142 of file nvdsinfer_context.h.

Function Documentation

NvDsInferStatus NvDsInferContext_Create ( NvDsInferContextHandle *  handle,
NvDsInferContextInitParams *  initParams,
void *  userCtx,
NvDsInferContextLoggingFunc  logFunc 
)

Creates a new instance of the NvDsInferContext class with the supplied initialization parameters.

Parameters
[out]handlePointer to a NvDsInferContext handle.
[in]initParamsParameters to use for initialization of the context.
[in]userCtxPointer to an opaque user context supplied with callbacks generated by the NvDsInferContext instance.
[in]logFuncLog callback function for the instance.
Returns
NVDSINFER_SUCCESS if the creation succeeded or a relevant error status.
NvDsInferStatus NvDsInferContext_DequeueOutputBatch ( NvDsInferContextHandle  handle,
NvDsInferContextBatchOutput *  batchOutput 
)

Dequeue output for a batch of frames.

Refer NvDsInferContext::dequeueOutputBatch() for details.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in,out]batchOutputReference to the Batch output structure in which the output will be appended.
Returns
NVDSINFER_SUCCESS if dequeueing succeeded or a relevant error status.
void NvDsInferContext_Destroy ( NvDsInferContextHandle  handle)

Destroy a NvDsInferContext instance and release the associated resources.

Parameters
[in]handleHandle to the NvDsInferContext instance to be destroyed.
void NvDsInferContext_FillLayersInfo ( NvDsInferContextHandle  handle,
NvDsInferLayerInfo *  layersInfo 
)

Fill the input vector with information on all the bound layers of the inference engine in the NvDsInferContext instance.

The size of the array has to be at least the value returned by NvDsInferContext_GetNumLayersInfo.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in,out]layersInfoPointer to an array of NvDsInferLayerInfo which will be filled by the function.
const char* NvDsInferContext_GetLabel ( NvDsInferContextHandle  handle,
unsigned int  id,
unsigned int  value 
)

Get the string label associated with the class_id for detectors and the attribute id and the attribute value for classifiers.

The string is owned by the context. Callers should not modify or free the string.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in]idClass ID for detectors, Attribute ID for classifiers.
[in]valueAttribute value for classifiers, should be 0 for detectors.
Returns
Pointer to a string label. The memory is owned by the context.
void NvDsInferContext_GetNetworkInfo ( NvDsInferContextHandle  handle,
NvDsInferNetworkInfo *  networkInfo 
)

Get the network input information.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in,out]networkInfoPointer to a NvDsInferNetworkInfo structure.
unsigned int NvDsInferContext_GetNumLayersInfo ( NvDsInferContextHandle  handle)

Get the number of the bound layers of the inference engine in the NvDsInferContext instance.

Parameters
[in]handleHandle to a NvDsInferContext instance.
Returns
The number of the bound layers of the inference engine.
const char* NvDsInferContext_GetStatusName ( NvDsInferStatus  status)

Get the string name for the status.

Parameters
[in]statusAn NvDsInferStatus value.
Returns
String name for the status. NULL in case of an unrecognized status value. Memory is owned by the function. Callers should not free the pointer.
NvDsInferStatus NvDsInferContext_QueueInputBatch ( NvDsInferContextHandle  handle,
NvDsInferContextBatchInput *  batchInput 
)

Queue a batch of input frames for pre-processing and inferencing.

Refer NvDsInferContext::queueInputBatch() for details.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in]batchInputReference to a batch input structure.
Returns
NVDSINFER_SUCCESS if pre-processing and queueing succeeded or a relevant error status.
void NvDsInferContext_ReleaseBatchOutput ( NvDsInferContextHandle  handle,
NvDsInferContextBatchOutput *  batchOutput 
)

Free the memory associated with the batch output and release the set of host buffers back to the context for re-use.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in]batchOutputPointer to a NvDsInferContext_ReleaseBatchOutput structure which was filled by NvDsInferContext_DequeueOutputBatch().
void NvDsInferContext_ResetInitParams ( NvDsInferContextInitParams *  initParams)

Reset the members of initialization parameters to default values.

Parameters
[in]initParamsPointer to a NvDsInferContextInitParams structure.