NVIDIA DeepStream SDK API Reference

4.0.1 Release

 All Data Structures Files Functions Variables Typedefs Enumerations Enumerator Groups
Inference Interface API

Detailed Description

Defines the DeepStream inference interface API.

In C++, defines the NvDsInferContext class.

Data Structures

struct  NvDsInferDetectionParams
 Holds the detection and bounding box grouping parameters. More...
 
struct  _NvDsInferContextInitParams
 Holds the initialization parameters required for the NvDsInferContext interface. More...
 
struct  NvDsInferContextBatchInput
 Holds the information about one batch to be inferred. More...
 
struct  NvDsInferObject
 Holds the information about one detected object. More...
 
struct  NvDsInferDetectionOutput
 Holds the information on all objects detected by a detector network in one frame. More...
 
struct  NvDsInferClassificationOutput
 Holds the information on all attributes classifed by a classifier network for one frame. More...
 
struct  NvDsInferSegmentationOutput
 Holds the information parsed from segmentation network output for one frame. More...
 
struct  NvDsInferFrameOutput
 Holds the information inferred by the network on one frame. More...
 
struct  NvDsInferContextBatchOutput
 A batch is an array of frames. More...
 

NvDsInferContext API common types and functions.

This section describes the common types and functions for both the C and C++ interfaces for the NvDsInferContext class.

enum  NvDsInferNetworkMode {
  NvDsInferNetworkMode_FP32,
  NvDsInferNetworkMode_INT8,
  NvDsInferNetworkMode_FP16
}
 Enum for internal data format to be used by the inference engine. More...
 
enum  NvDsInferNetworkType {
  NvDsInferNetworkType_Detector,
  NvDsInferNetworkType_Classifier,
  NvDsInferNetworkType_Segmentation,
  NvDsInferNetworkType_Other = 100
}
 Enum for the type of the network. More...
 
enum  NvDsInferFormat {
  NvDsInferFormat_RGB,
  NvDsInferFormat_BGR,
  NvDsInferFormat_GRAY,
  NvDsInferFormat_RGBA,
  NvDsInferFormat_BGRx,
  NvDsInferFormat_Unknown = 0xFFFFFFFF
}
 Enum for color formats. More...
 
enum  NvDsInferUffInputOrder {
  NvDsInferUffInputOrder_kNCHW,
  NvDsInferUffInputOrder_kNHWC,
  NvDsInferUffInputOrder_kNC
}
 Enum for the UFF input layer order. More...
 
enum  NvDsInferStatus {
  NVDSINFER_SUCCESS = 0,
  NVDSINFER_CONFIG_FAILED,
  NVDSINFER_CUSTOM_LIB_FAILED,
  NVDSINFER_INVALID_PARAMS,
  NVDSINFER_OUTPUT_PARSING_FAILED,
  NVDSINFER_CUDA_ERROR,
  NVDSINFER_TENSORRT_ERROR,
  NVDSINFER_UNKNOWN_ERROR
}
 Enum for the status codes returned by NvDsInferContext. More...
 
enum  NvDsInferLogLevel {
  NVDSINFER_LOG_ERROR,
  NVDSINFER_LOG_WARNING,
  NVDSINFER_LOG_INFO,
  NVDSINFER_LOG_DEBUG
}
 Enum for the log levels of NvDsInferContext. More...
 
typedef struct
_NvDsInferContextInitParams 
NvDsInferContextInitParams
 Holds the initialization parameters required for the NvDsInferContext interface. More...
 
typedef void(* NvDsInferContextReturnInputAsyncFunc )(void *data)
 Callback function type for returning the input client buffers back to the NvDsInferContext client asynchronously. More...
 
typedef struct INvDsInferContext * NvDsInferContextHandle
 An opaque pointer type to be used as an handle for the context instance. More...
 
typedef void(* NvDsInferContextLoggingFunc )(NvDsInferContextHandle handle, unsigned int uniqueID, NvDsInferLogLevel logLevel, const char *funcName, const char *logMessage, void *userCtx)
 Callback function type for logging the NvDsInferContext messages. More...
 
void NvDsInferContext_ResetInitParams (NvDsInferContextInitParams *initParams)
 Reset the members of initialization parameters to default values. More...
 
const char * NvDsInferContext_GetStatusName (NvDsInferStatus status)
 Get the string name for the status. More...
 
#define _PATH_MAX   4096
 Maximum length of a file path parameter. More...
 
#define _MAX_CHANNELS   4
 Maximum number of channels supported by the API for image input layers. More...
 
#define _MAX_STR_LENGTH   1024
 Maximum length of string parameters. More...
 
#define NVDSINFER_MAX_BATCH_SIZE   1024
 Maximum batch size to be supported by nvdsinfer. More...
 
#define NVDSINFER_MIN_OUTPUT_BUFFERPOOL_SIZE   2
 Minimum number of sets of output buffers that need to be allocated. More...
 

NvDsInferContext API C-interface

This section describes the C interface for the NvDsInferContext class.

NvDsInferStatus NvDsInferContext_Create (NvDsInferContextHandle *handle, NvDsInferContextInitParams *initParams, void *userCtx, NvDsInferContextLoggingFunc logFunc)
 Creates a new instance of the NvDsInferContext class with the supplied initialization parameters. More...
 
void NvDsInferContext_Destroy (NvDsInferContextHandle handle)
 Destroy a NvDsInferContext instance and release the associated resources. More...
 
NvDsInferStatus NvDsInferContext_QueueInputBatch (NvDsInferContextHandle handle, NvDsInferContextBatchInput *batchInput)
 Queue a batch of input frames for pre-processing and inferencing. More...
 
NvDsInferStatus NvDsInferContext_DequeueOutputBatch (NvDsInferContextHandle handle, NvDsInferContextBatchOutput *batchOutput)
 Dequeue output for a batch of frames. More...
 
void NvDsInferContext_ReleaseBatchOutput (NvDsInferContextHandle handle, NvDsInferContextBatchOutput *batchOutput)
 Free the memory associated with the batch output and release the set of host buffers back to the context for re-use. More...
 
void NvDsInferContext_GetNetworkInfo (NvDsInferContextHandle handle, NvDsInferNetworkInfo *networkInfo)
 Get the network input information. More...
 
unsigned int NvDsInferContext_GetNumLayersInfo (NvDsInferContextHandle handle)
 Get the number of the bound layers of the inference engine in the NvDsInferContext instance. More...
 
void NvDsInferContext_FillLayersInfo (NvDsInferContextHandle handle, NvDsInferLayerInfo *layersInfo)
 Fill the input vector with information on all the bound layers of the inference engine in the NvDsInferContext instance. More...
 
const char * NvDsInferContext_GetLabel (NvDsInferContextHandle handle, unsigned int id, unsigned int value)
 Get the string label associated with the class_id for detectors and the attribute id and the attribute value for classifiers. More...
 

Macro Definition Documentation

#define _MAX_CHANNELS   4

Maximum number of channels supported by the API for image input layers.

Definition at line 96 of file nvdsinfer_context.h.

#define _MAX_STR_LENGTH   1024

Maximum length of string parameters.

Definition at line 99 of file nvdsinfer_context.h.

#define _PATH_MAX   4096

Maximum length of a file path parameter.

Definition at line 93 of file nvdsinfer_context.h.

#define NVDSINFER_MAX_BATCH_SIZE   1024

Maximum batch size to be supported by nvdsinfer.

Definition at line 102 of file nvdsinfer_context.h.

#define NVDSINFER_MIN_OUTPUT_BUFFERPOOL_SIZE   2

Minimum number of sets of output buffers that need to be allocated.

Definition at line 105 of file nvdsinfer_context.h.

Typedef Documentation

typedef struct INvDsInferContext* NvDsInferContextHandle

An opaque pointer type to be used as an handle for the context instance.

Definition at line 480 of file nvdsinfer_context.h.

Holds the initialization parameters required for the NvDsInferContext interface.

typedef void(* NvDsInferContextLoggingFunc)(NvDsInferContextHandle handle, unsigned int uniqueID, NvDsInferLogLevel logLevel, const char *funcName, const char *logMessage, void *userCtx)

Callback function type for logging the NvDsInferContext messages.

Parameters
[in]handleHandle of the NvDsInferContext instance which generated the log.
[in]uniqueIDUniqueID of the NvDsInferContext instance which generated the log.
[in]logLevelLevel of the log.
[in]funcNameName of the function which generated the log.
[in]logMessageThe log message string.
[in]userCtxThe opaque user context pointer supplied when creating the NvDsInferContext instance.

Definition at line 494 of file nvdsinfer_context.h.

typedef void(* NvDsInferContextReturnInputAsyncFunc)(void *data)

Callback function type for returning the input client buffers back to the NvDsInferContext client asynchronously.

Parameters
[in]dataThe opaque pointer provided to input queueing function through NvDsInferContextBatchInput.

Definition at line 347 of file nvdsinfer_context.h.

Enumeration Type Documentation

Enum for color formats.

Enumerator
NvDsInferFormat_RGB 

24-bit interleaved R-G-B

NvDsInferFormat_BGR 

24-bit interleaved B-G-R

NvDsInferFormat_GRAY 

8-bit Luma

NvDsInferFormat_RGBA 

32-bit interleaved R-G-B-A

NvDsInferFormat_BGRx 

32-bit interleaved B-G-R-x

NvDsInferFormat_Unknown 

Definition at line 139 of file nvdsinfer_context.h.

Enum for the log levels of NvDsInferContext.

Enumerator
NVDSINFER_LOG_ERROR 
NVDSINFER_LOG_WARNING 
NVDSINFER_LOG_INFO 
NVDSINFER_LOG_DEBUG 

Definition at line 210 of file nvdsinfer_context.h.

Enum for internal data format to be used by the inference engine.

Enumerator
NvDsInferNetworkMode_FP32 
NvDsInferNetworkMode_INT8 
NvDsInferNetworkMode_FP16 

Definition at line 110 of file nvdsinfer_context.h.

Enum for the type of the network.

Enumerator
NvDsInferNetworkType_Detector 

Detectors will find objects and their coordinates in the input frame along with the class of the object.

NvDsInferNetworkType_Classifier 

Classifiers - will classify the entire frame into some finite possible classes.

NvDsInferNetworkType_Segmentation 

Segmentation - will classify each pixel into some finite possible classes.

NvDsInferNetworkType_Other 

Other - Output layers will not be parsed by NvDsInferContext.

This is useful for networks producing custom output. The output can be parsed by the NvDsInferContext client or can be combined with "gst-nvinfer" feature to flow output tensors as metadata.

Definition at line 120 of file nvdsinfer_context.h.

Enum for the status codes returned by NvDsInferContext.

Enumerator
NVDSINFER_SUCCESS 

NvDsInferContext operation succeeded.

NVDSINFER_CONFIG_FAILED 

Failed to configure the NvDsInferContext instance possibly due to an erroneous initialization property.

NVDSINFER_CUSTOM_LIB_FAILED 

Custom Library interface implementation failed.

NVDSINFER_INVALID_PARAMS 

Invalid parameters were supplied.

NVDSINFER_OUTPUT_PARSING_FAILED 

Output parsing failed.

NVDSINFER_CUDA_ERROR 

CUDA error was encountered.

NVDSINFER_TENSORRT_ERROR 

TensorRT interface failed.

NVDSINFER_UNKNOWN_ERROR 

Unknown error was encountered.

Definition at line 186 of file nvdsinfer_context.h.

Enum for the UFF input layer order.

Enumerator
NvDsInferUffInputOrder_kNCHW 
NvDsInferUffInputOrder_kNHWC 
NvDsInferUffInputOrder_kNC 

Definition at line 157 of file nvdsinfer_context.h.

Function Documentation

NvDsInferStatus NvDsInferContext_Create ( NvDsInferContextHandle handle,
NvDsInferContextInitParams initParams,
void *  userCtx,
NvDsInferContextLoggingFunc  logFunc 
)

Creates a new instance of the NvDsInferContext class with the supplied initialization parameters.

Parameters
[out]handlePointer to a NvDsInferContext handle.
[in]initParamsParameters to use for initialization of the context.
[in]userCtxPointer to an opaque user context supplied with callbacks generated by the NvDsInferContext instance.
[in]logFuncLog callback function for the instance.
Returns
NVDSINFER_SUCCESS if the creation succeeded or a relevant error status.
NvDsInferStatus NvDsInferContext_DequeueOutputBatch ( NvDsInferContextHandle  handle,
NvDsInferContextBatchOutput batchOutput 
)

Dequeue output for a batch of frames.

Refer NvDsInferContext::dequeueOutputBatch() for details.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in,out]batchOutputReference to the Batch output structure in which the output will be appended.
Returns
NVDSINFER_SUCCESS if dequeueing succeeded or a relevant error status.
void NvDsInferContext_Destroy ( NvDsInferContextHandle  handle)

Destroy a NvDsInferContext instance and release the associated resources.

Parameters
[in]handleHandle to the NvDsInferContext instance to be destroyed.
void NvDsInferContext_FillLayersInfo ( NvDsInferContextHandle  handle,
NvDsInferLayerInfo layersInfo 
)

Fill the input vector with information on all the bound layers of the inference engine in the NvDsInferContext instance.

The size of the array has to be at least the value returned by NvDsInferContext_GetNumLayersInfo.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in,out]layersInfoPointer to an array of NvDsInferLayerInfo which will be filled by the function.
const char* NvDsInferContext_GetLabel ( NvDsInferContextHandle  handle,
unsigned int  id,
unsigned int  value 
)

Get the string label associated with the class_id for detectors and the attribute id and the attribute value for classifiers.

The string is owned by the context. Callers should not modify or free the string.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in]idClass ID for detectors, Attribute ID for classifiers.
[in]valueAttribute value for classifiers, should be 0 for detectors.
Returns
Pointer to a string label. The memory is owned by the context.
void NvDsInferContext_GetNetworkInfo ( NvDsInferContextHandle  handle,
NvDsInferNetworkInfo networkInfo 
)

Get the network input information.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in,out]networkInfoPointer to a NvDsInferNetworkInfo structure.
unsigned int NvDsInferContext_GetNumLayersInfo ( NvDsInferContextHandle  handle)

Get the number of the bound layers of the inference engine in the NvDsInferContext instance.

Parameters
[in]handleHandle to a NvDsInferContext instance.
Returns
The number of the bound layers of the inference engine.
const char* NvDsInferContext_GetStatusName ( NvDsInferStatus  status)

Get the string name for the status.

Parameters
[in]statusAn NvDsInferStatus value.
Returns
String name for the status. NULL in case of an unrecognized status value. Memory is owned by the function. Callers should not free the pointer.
NvDsInferStatus NvDsInferContext_QueueInputBatch ( NvDsInferContextHandle  handle,
NvDsInferContextBatchInput batchInput 
)

Queue a batch of input frames for pre-processing and inferencing.

Refer NvDsInferContext::queueInputBatch() for details.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in]batchInputReference to a batch input structure.
Returns
NVDSINFER_SUCCESS if pre-processing and queueing succeeded or a relevant error status.
void NvDsInferContext_ReleaseBatchOutput ( NvDsInferContextHandle  handle,
NvDsInferContextBatchOutput batchOutput 
)

Free the memory associated with the batch output and release the set of host buffers back to the context for re-use.

Parameters
[in]handleHandle to a NvDsInferContext instance.
[in]batchOutputPointer to a NvDsInferContext_ReleaseBatchOutput structure which was filled by NvDsInferContext_DequeueOutputBatch().
void NvDsInferContext_ResetInitParams ( NvDsInferContextInitParams initParams)

Reset the members of initialization parameters to default values.

Parameters
[in]initParamsPointer to a NvDsInferContextInitParams structure.