Detailed Description

Definition at line 60 of file infer_ibackend.h.

Inheritance diagram for nvdsinferserver::IBackend:

[legend]

Public Types
enum	{ kLTpLayerDesc, kTpLayerNum }

enum	{ kInShapeName, kInShapeDims }

using	InferenceDone = std::function< void(NvDsInferStatus, SharedBatchArray)>
	Function wrapper for post inference processing. More...

using	InputsConsumed = std::function< void(SharedBatchArray)>
	Function wrapper called after the input buffer is consumed. More...

using	LayersTuple = std::tuple< const LayerDescription *, int >
	Tuple containing pointer to layer descriptions and the number of layers. More...

using	InputShapeTuple = std::tuple< std::string, InferBatchDims >
	Tuple of layer name and dimensions including batch size. More...

using	InputShapes = std::vector< InputShapeTuple >

Public Member Functions
	IBackend ()=default
	Constructor, default. More...

virtual	~IBackend ()=default
	Destructor, default. More...

virtual NvDsInferStatus	initialize ()=0
	Initialize the backend for processing. More...

virtual NvDsInferStatus	specifyInputDims (const InputShapes &shapes)=0
	Specify the input layers for the backend. More...

virtual bool	isFirstDimBatch () const =0
	Check if the flag for first dimension being batch is set. More...

virtual InferTensorOrder	getInputTensorOrder () const =0
	Get the tensor order set for the input. More...

virtual int32_t	maxBatchSize () const =0
	Get the configured maximum batch size for this backend. More...

virtual uint32_t	getLayerSize () const =0
	Get the number of layers (input and output) for the model. More...

virtual uint32_t	getInputLayerSize () const =0
	Get the number of input layers. More...

virtual const LayerDescription *	getLayerInfo (const std::string &bindingName) const =0
	Get the layer description from the layer name. More...

virtual LayersTuple	getInputLayers () const =0
	Get the LayersTuple for input layers. More...

virtual LayersTuple	getOutputLayers () const =0
	Get the LayersTuple for output layers. More...

virtual NvDsInferStatus	enqueue (SharedBatchArray inputs, SharedCuStream stream, InputsConsumed bufConsumed, InferenceDone inferenceDone)=0
	Enqueue an array of input batches for inference. More...

Member Typedef Documentation

◆ InferenceDone

using nvdsinferserver::IBackend::InferenceDone = std::function<void(NvDsInferStatus, SharedBatchArray)>

Function wrapper for post inference processing.

Definition at line 66 of file infer_ibackend.h.

◆ InputsConsumed

using nvdsinferserver::IBackend::InputsConsumed = std::function<void(SharedBatchArray)>

Function wrapper called after the input buffer is consumed.

Definition at line 70 of file infer_ibackend.h.

◆ InputShapes

using nvdsinferserver::IBackend::InputShapes = std::vector<InputShapeTuple>

Definition at line 84 of file infer_ibackend.h.

◆ InputShapeTuple

using nvdsinferserver::IBackend::InputShapeTuple = std::tuple<std::string, InferBatchDims>

Tuple of layer name and dimensions including batch size.

Definition at line 83 of file infer_ibackend.h.

◆ LayersTuple

using nvdsinferserver::IBackend::LayersTuple = std::tuple<const LayerDescription*, int>

Tuple containing pointer to layer descriptions and the number of layers.

Definition at line 77 of file infer_ibackend.h.

Member Enumeration Documentation

◆ anonymous enum

anonymous enum

Enumerator
kLTpLayerDesc
kTpLayerNum

Definition at line 72 of file infer_ibackend.h.

◆ anonymous enum

anonymous enum

Enumerator
kInShapeName
kInShapeDims

Definition at line 79 of file infer_ibackend.h.

Constructor & Destructor Documentation

◆ IBackend()

nvdsinferserver::IBackend::IBackend ( )

default

Constructor, default.

◆ ~IBackend()

virtual nvdsinferserver::IBackend::~IBackend ( )

virtualdefault

Destructor, default.

Member Function Documentation

◆ enqueue()

virtual NvDsInferStatus nvdsinferserver::IBackend::enqueue	(	SharedBatchArray	inputs,
		SharedCuStream	stream,
		InputsConsumed	bufConsumed,
		InferenceDone	inferenceDone
	)

pure virtual

Enqueue an array of input batches for inference.

This function adds a input to the inference processing queue of the backend. The post inference function and function to be called after consuming input buffer is provided.

Parameters

[in]	inputs	List of input batch buffers
[in]	stream	The CUDA stream to be used in inference processing.
[in]	bufConsumed	Function to be called once input buffer is consumed.
[in]	inferenceDone	Function to be called after inference.

Returns: Execution status code.

Implemented in nvdsinferserver::TrtISBackend, nvdsinferserver::TritonGrpcBackend, and nvdsinferserver::TritonSimpleRuntime.

◆ getInputLayers()

virtual LayersTuple nvdsinferserver::IBackend::getInputLayers ( ) const

pure virtual

Get the LayersTuple for input layers.

Implemented in nvdsinferserver::BaseBackend.

◆ getInputLayerSize()

virtual uint32_t nvdsinferserver::IBackend::getInputLayerSize ( ) const

pure virtual

Get the number of input layers.

Implemented in nvdsinferserver::BaseBackend.

◆ getInputTensorOrder()

virtual InferTensorOrder nvdsinferserver::IBackend::getInputTensorOrder ( ) const

pure virtual

Get the tensor order set for the input.

Implemented in nvdsinferserver::BaseBackend.

◆ getLayerInfo()

virtual const LayerDescription* nvdsinferserver::IBackend::getLayerInfo ( const std::string & bindingName ) const

pure virtual

Get the layer description from the layer name.

Implemented in nvdsinferserver::BaseBackend.

◆ getLayerSize()

virtual uint32_t nvdsinferserver::IBackend::getLayerSize ( ) const

pure virtual

Get the number of layers (input and output) for the model.

Implemented in nvdsinferserver::BaseBackend.

◆ getOutputLayers()

virtual LayersTuple nvdsinferserver::IBackend::getOutputLayers ( ) const

pure virtual

Get the LayersTuple for output layers.

Implemented in nvdsinferserver::BaseBackend.

◆ initialize()

virtual NvDsInferStatus nvdsinferserver::IBackend::initialize ( )

pure virtual

Initialize the backend for processing.

Returns: Status code of the type NvDsInferStatus.

Implemented in nvdsinferserver::TrtISBackend, nvdsinferserver::TritonGrpcBackend, and nvdsinferserver::TritonSimpleRuntime.

◆ isFirstDimBatch()

virtual bool nvdsinferserver::IBackend::isFirstDimBatch ( ) const

pure virtual

Check if the flag for first dimension being batch is set.

Implemented in nvdsinferserver::BaseBackend.

◆ maxBatchSize()

virtual int32_t nvdsinferserver::IBackend::maxBatchSize ( ) const

pure virtual

Get the configured maximum batch size for this backend.

Implemented in nvdsinferserver::BaseBackend.

◆ specifyInputDims()

virtual NvDsInferStatus nvdsinferserver::IBackend::specifyInputDims ( const InputShapes & shapes )

pure virtual

Specify the input layers for the backend.

Parameters

shapes List of name and shapes of the input layers.

Returns: Status code of the type NvDsInferStatus.

Implemented in nvdsinferserver::TrtISBackend, and nvdsinferserver::TritonSimpleRuntime.

The documentation for this class was generated from the following file:

infer_ibackend.h

NVIDIA DeepStream SDK API Reference

6.4 Release

Detailed Description

Public Types

Public Member Functions

Member Typedef Documentation

◆ InferenceDone

◆ InputsConsumed

◆ InputShapes

◆ InputShapeTuple

◆ LayersTuple

Member Enumeration Documentation

◆ anonymous enum

◆ anonymous enum

Constructor & Destructor Documentation

◆ IBackend()

◆ ~IBackend()

Member Function Documentation

◆ enqueue()

◆ getInputLayers()

◆ getInputLayerSize()

◆ getInputTensorOrder()

◆ getLayerInfo()

◆ getLayerSize()

◆ getOutputLayers()

◆ initialize()

◆ isFirstDimBatch()

◆ maxBatchSize()

◆ specifyInputDims()