NVIDIA DeepStream SDK API Reference

4.0.1 Release

 All Data Structures Files Functions Variables Typedefs Enumerations Enumerator Groups
nvdsinfer_context.h File Reference

Detailed Description

NVIDIA DeepStream Inference Interface

Description: This file specifies the DeepStream inference interface API.

The Gst-nvinfer API provides methods to initialize and deinitialize the inference engine, preprocess input frames as required by the network, and parse output from the raw tensor buffers.

Both C and C++ interfaces are available with the C interface being a simple wrapper over the C++ interface.

An opaque handle (NvDsInferContext) to an instance of the context required by the APIs can be created using factory functions createNvDsInferContext / NvDsInferContext_Create. The two functions accept an instance of NvDsInferContextInitParams structure to initialize the context. Logging callbacks can be set to get detailed information about failures/warnings.

Initialization parameters allow configuration of the network data type, type of the network(Detector/Classifier/Other), pre-processing parameters (mean subtraction and normalization), model related parameters like Caffe/Uff/Onnx model file paths, output layer names etc.

Batches of frames can be queued for inferencing using NvDsInferContext::queueInputBatch / NvDsInferContext_QueueInputBatch. The input frame memories should be accessible by the GPU device configured during initialization. An asynchronous callback function can be provided to return the input buffers back to the caller as soon as the input is consumed.

Inference output can be dequeued using NvDsInferContext::dequeueOutputBatch / NvDsInferContext_DequeueOutputBatch. The order dequeued outputs corresponds to the input queueing order. In case of failure, the output of the batch is lost. The dequeued output must be released back to the context using NvDsInferContext::releaseBatchOutput / NvDsInferContext_ReleaseBatchOutput to free the associated memory and return the output layer buffers for re-use by the context.

Detectors will output an array of detected objects for each frame in the batch. Classifiers will classify entire frames and output an array of attributes for each frame. Segmentation will classify each pixel in the frame. A special network type has been provided (Other) where the output layers will not be parsed. The caller can parse the device/host output layer buffers. This can also be used along with DeepStream GStreamer Inference plugin "nvinfer" to flow the output buffers as metadata.

Other methods to get the parsed labels from a labels file and properties of all the layers bound by the inferece engine have been provided.

The NvDsInfer APIs can be extended using the custom method implementations. Refer to the Custom Method Implementations section for more details.

Definition in file nvdsinfer_context.h.

Go to the source code of this file.

Data Structures

struct  NvDsInferDetectionParams
 Holds the detection and bounding box grouping parameters. More...
 
struct  _NvDsInferContextInitParams
 Holds the initialization parameters required for the NvDsInferContext interface. More...
 
struct  NvDsInferContextBatchInput
 Holds the information about one batch to be inferred. More...
 
struct  NvDsInferObject
 Holds the information about one detected object. More...
 
struct  NvDsInferDetectionOutput
 Holds the information on all objects detected by a detector network in one frame. More...
 
struct  NvDsInferClassificationOutput
 Holds the information on all attributes classifed by a classifier network for one frame. More...
 
struct  NvDsInferSegmentationOutput
 Holds the information parsed from segmentation network output for one frame. More...
 
struct  NvDsInferFrameOutput
 Holds the information inferred by the network on one frame. More...
 
struct  NvDsInferContextBatchOutput
 A batch is an array of frames. More...
 

Functions

NvDsInferContext API C-interface

This section describes the C interface for the NvDsInferContext class.

NvDsInferStatus NvDsInferContext_Create (NvDsInferContextHandle *handle, NvDsInferContextInitParams *initParams, void *userCtx, NvDsInferContextLoggingFunc logFunc)
 Creates a new instance of the NvDsInferContext class with the supplied initialization parameters. More...
 
void NvDsInferContext_Destroy (NvDsInferContextHandle handle)
 Destroy a NvDsInferContext instance and release the associated resources. More...
 
NvDsInferStatus NvDsInferContext_QueueInputBatch (NvDsInferContextHandle handle, NvDsInferContextBatchInput *batchInput)
 Queue a batch of input frames for pre-processing and inferencing. More...
 
NvDsInferStatus NvDsInferContext_DequeueOutputBatch (NvDsInferContextHandle handle, NvDsInferContextBatchOutput *batchOutput)
 Dequeue output for a batch of frames. More...
 
void NvDsInferContext_ReleaseBatchOutput (NvDsInferContextHandle handle, NvDsInferContextBatchOutput *batchOutput)
 Free the memory associated with the batch output and release the set of host buffers back to the context for re-use. More...
 
void NvDsInferContext_GetNetworkInfo (NvDsInferContextHandle handle, NvDsInferNetworkInfo *networkInfo)
 Get the network input information. More...
 
unsigned int NvDsInferContext_GetNumLayersInfo (NvDsInferContextHandle handle)
 Get the number of the bound layers of the inference engine in the NvDsInferContext instance. More...
 
void NvDsInferContext_FillLayersInfo (NvDsInferContextHandle handle, NvDsInferLayerInfo *layersInfo)
 Fill the input vector with information on all the bound layers of the inference engine in the NvDsInferContext instance. More...
 
const char * NvDsInferContext_GetLabel (NvDsInferContextHandle handle, unsigned int id, unsigned int value)
 Get the string label associated with the class_id for detectors and the attribute id and the attribute value for classifiers. More...
 

NvDsInferContext API common types and functions.

This section describes the common types and functions for both the C and C++ interfaces for the NvDsInferContext class.

#define _PATH_MAX   4096
 Maximum length of a file path parameter. More...
 
#define _MAX_CHANNELS   4
 Maximum number of channels supported by the API for image input layers. More...
 
#define _MAX_STR_LENGTH   1024
 Maximum length of string parameters. More...
 
#define NVDSINFER_MAX_BATCH_SIZE   1024
 Maximum batch size to be supported by nvdsinfer. More...
 
#define NVDSINFER_MIN_OUTPUT_BUFFERPOOL_SIZE   2
 Minimum number of sets of output buffers that need to be allocated. More...
 
enum  NvDsInferNetworkMode {
  NvDsInferNetworkMode_FP32,
  NvDsInferNetworkMode_INT8,
  NvDsInferNetworkMode_FP16
}
 Enum for internal data format to be used by the inference engine. More...
 
enum  NvDsInferNetworkType {
  NvDsInferNetworkType_Detector,
  NvDsInferNetworkType_Classifier,
  NvDsInferNetworkType_Segmentation,
  NvDsInferNetworkType_Other = 100
}
 Enum for the type of the network. More...
 
enum  NvDsInferFormat {
  NvDsInferFormat_RGB,
  NvDsInferFormat_BGR,
  NvDsInferFormat_GRAY,
  NvDsInferFormat_RGBA,
  NvDsInferFormat_BGRx,
  NvDsInferFormat_Unknown = 0xFFFFFFFF
}
 Enum for color formats. More...
 
enum  NvDsInferUffInputOrder {
  NvDsInferUffInputOrder_kNCHW,
  NvDsInferUffInputOrder_kNHWC,
  NvDsInferUffInputOrder_kNC
}
 Enum for the UFF input layer order. More...
 
enum  NvDsInferStatus {
  NVDSINFER_SUCCESS = 0,
  NVDSINFER_CONFIG_FAILED,
  NVDSINFER_CUSTOM_LIB_FAILED,
  NVDSINFER_INVALID_PARAMS,
  NVDSINFER_OUTPUT_PARSING_FAILED,
  NVDSINFER_CUDA_ERROR,
  NVDSINFER_TENSORRT_ERROR,
  NVDSINFER_UNKNOWN_ERROR
}
 Enum for the status codes returned by NvDsInferContext. More...
 
enum  NvDsInferLogLevel {
  NVDSINFER_LOG_ERROR,
  NVDSINFER_LOG_WARNING,
  NVDSINFER_LOG_INFO,
  NVDSINFER_LOG_DEBUG
}
 Enum for the log levels of NvDsInferContext. More...
 
typedef struct
_NvDsInferContextInitParams 
NvDsInferContextInitParams
 Holds the initialization parameters required for the NvDsInferContext interface. More...
 
typedef void(* NvDsInferContextReturnInputAsyncFunc )(void *data)
 Callback function type for returning the input client buffers back to the NvDsInferContext client asynchronously. More...
 
typedef struct INvDsInferContext * NvDsInferContextHandle
 An opaque pointer type to be used as an handle for the context instance. More...
 
typedef void(* NvDsInferContextLoggingFunc )(NvDsInferContextHandle handle, unsigned int uniqueID, NvDsInferLogLevel logLevel, const char *funcName, const char *logMessage, void *userCtx)
 Callback function type for logging the NvDsInferContext messages. More...
 
void NvDsInferContext_ResetInitParams (NvDsInferContextInitParams *initParams)
 Reset the members of initialization parameters to default values. More...
 
const char * NvDsInferContext_GetStatusName (NvDsInferStatus status)
 Get the string name for the status. More...