Detailed Description

Wrapper class for Triton inference request.

Definition at line 68 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

Public Member Functions
	~TrtServerRequest ()
	Destructor. More...

TRITONSERVER_InferenceRequest *	ptr ()
	Get the pointer to the Triton inference request object. More...

const std::string &	model () const
	Get the model name. More...

uint64_t	id () const
	Get the request ID. More...

uint64_t	bufId () const
	Get the input buffer ID associated with the request. More...

SharedBatchArray	releaseInputs ()
	Release the ownership of input batch buffer array. More...

const std::vector< std::string > &	outputs () const
	Get the list of requested output layer names. More...

const std::map< std::string, TritonClassParams > &	classParams () const
	Get the Triton classification parameters list (tensor name : classification parameters). More...

	~TrtServerRequest ()
	Destructor. More...

TRITONSERVER_InferenceRequest *	ptr ()
	Get the pointer to the Triton inference request object. More...

const std::string &	model () const
	Get the model name. More...

uint64_t	id () const
	Get the request ID. More...

uint64_t	bufId () const
	Get the input buffer ID associated with the request. More...

SharedBatchArray	releaseInputs ()
	Release the ownership of input batch buffer array. More...

const std::vector< std::string > &	outputs () const
	Get the list of requested output layer names. More...

const std::map< std::string, TritonClassParams > &	classParams () const
	Get the Triton classification parameters list (tensor name : classification parameters). More...

Protected Member Functions
	TrtServerRequest (TrtServerPtr server)
	Constructor. More...

NvDsInferStatus	init (const std::string &model, int64_t version, SharedBatchArray &inputs, const std::vector< std::string > &outputs, uint64_t reqId, const std::vector< TritonClassParams > &clasList)
	Create a new Triton inference request with the specified inputs and parameters. More...

NvDsInferStatus	setRequestComplete (TRITONSERVER_InferenceRequestReleaseFn_t requestCompleteCb, void *userPtr)
	Set the release callback function for the request. More...

NvDsInferStatus	setResponseComplete (ShrTritonAllocator &allocator, TRITONSERVER_InferenceResponseCompleteFn_t responseCompleteCb, void *responseUserPtr)
	Set the allocator and response callback for the request. More...

	TrtServerRequest (TrtServerPtr server)
	Constructor. More...

NvDsInferStatus	init (const std::string &model, int64_t version, SharedBatchArray &inputs, const std::vector< std::string > &outputs, uint64_t reqId, const std::vector< TritonClassParams > &clasList)
	Create a new Triton inference request with the specified inputs and parameters. More...

NvDsInferStatus	setRequestComplete (TRITONSERVER_InferenceRequestReleaseFn_t requestCompleteCb, void *userPtr)
	Set the release callback function for the request. More...

NvDsInferStatus	setResponseComplete (ShrTritonAllocator &allocator, TRITONSERVER_InferenceResponseCompleteFn_t responseCompleteCb, void *responseUserPtr)
	Set the allocator and response callback for the request. More...

Static Protected Member Functions
static void	RequestOnRelease (TRITONSERVER_InferenceRequest request, const uint32_t flags, void userp)
	The callback function to release the request instance. More...

static void	RequestOnRelease (TRITONSERVER_InferenceRequest request, const uint32_t flags, void userp)
	The callback function to release the request instance. More...

Friends
class	TrtISServer

Constructor & Destructor Documentation

◆ TrtServerRequest() [1/2]

nvdsinferserver::TrtServerRequest::TrtServerRequest ( TrtServerPtr server )

protected

Constructor.

Save the server instance pointer and register the Triton request deletion function.

Parameters

server

◆ ~TrtServerRequest() [1/2]

nvdsinferserver::TrtServerRequest::~TrtServerRequest ( )

Destructor.

Releases the Triton inference request instance.

◆ TrtServerRequest() [2/2]

nvdsinferserver::TrtServerRequest::TrtServerRequest ( TrtServerPtr server )

protected

Constructor.

Save the server instance pointer and register the Triton request deletion function.

Parameters

server

◆ ~TrtServerRequest() [2/2]

nvdsinferserver::TrtServerRequest::~TrtServerRequest ( )

Destructor.

Releases the Triton inference request instance.

Member Function Documentation

◆ bufId() [1/2]

uint64_t nvdsinferserver::TrtServerRequest::bufId ( ) const

inline

Get the input buffer ID associated with the request.

Definition at line 147 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ bufId() [2/2]

uint64_t nvdsinferserver::TrtServerRequest::bufId ( ) const

inline

Get the input buffer ID associated with the request.

Definition at line 147 of file 9.0/sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ classParams() [1/2]

const std::map<std::string, TritonClassParams>& nvdsinferserver::TrtServerRequest::classParams ( ) const

inline

Get the Triton classification parameters list (tensor name : classification parameters).

Definition at line 161 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ classParams() [2/2]

const std::map<std::string, TritonClassParams>& nvdsinferserver::TrtServerRequest::classParams ( ) const

inline

Get the Triton classification parameters list (tensor name : classification parameters).

Definition at line 161 of file 9.0/sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ id() [1/2]

uint64_t nvdsinferserver::TrtServerRequest::id ( ) const

inline

Get the request ID.

Definition at line 143 of file 9.0/sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ id() [2/2]

uint64_t nvdsinferserver::TrtServerRequest::id ( ) const

inline

Get the request ID.

Definition at line 143 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ init() [1/2]

NvDsInferStatus nvdsinferserver::TrtServerRequest::init	(	const std::string &	model,
		int64_t	version,
		SharedBatchArray &	inputs,
		const std::vector< std::string > &	outputs,
		uint64_t	reqId,
		const std::vector< TritonClassParams > &	clasList
	)

protected

Create a new Triton inference request with the specified inputs and parameters.

Parameters

[in]	model	Model name.
[in]	version	Model version.
[in]	inputs	Array of input batch buffers.
[in]	outputs	List of names of required output tensors.
[in]	reqId	ID of this request.
[in]	clasList	Triton classification parameters, if any.

Returns: NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

◆ init() [2/2]

NvDsInferStatus nvdsinferserver::TrtServerRequest::init	(	const std::string &	model,
		int64_t	version,
		SharedBatchArray &	inputs,
		const std::vector< std::string > &	outputs,
		uint64_t	reqId,
		const std::vector< TritonClassParams > &	clasList
	)

protected

Create a new Triton inference request with the specified inputs and parameters.

Parameters

[in]	model	Model name.
[in]	version	Model version.
[in]	inputs	Array of input batch buffers.
[in]	outputs	List of names of required output tensors.
[in]	reqId	ID of this request.
[in]	clasList	Triton classification parameters, if any.

Returns: NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

◆ model() [1/2]

const std::string& nvdsinferserver::TrtServerRequest::model ( ) const

inline

Get the model name.

Definition at line 139 of file 9.0/sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ model() [2/2]

const std::string& nvdsinferserver::TrtServerRequest::model ( ) const

inline

Get the model name.

Definition at line 139 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ outputs() [1/2]

const std::vector<std::string>& nvdsinferserver::TrtServerRequest::outputs ( ) const

inline

Get the list of requested output layer names.

Definition at line 156 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ outputs() [2/2]

const std::vector<std::string>& nvdsinferserver::TrtServerRequest::outputs ( ) const

inline

Get the list of requested output layer names.

Definition at line 156 of file 9.0/sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ ptr() [1/2]

TRITONSERVER_InferenceRequest* nvdsinferserver::TrtServerRequest::ptr ( )

inline

Get the pointer to the Triton inference request object.

Definition at line 135 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ ptr() [2/2]

TRITONSERVER_InferenceRequest* nvdsinferserver::TrtServerRequest::ptr ( )

inline

Get the pointer to the Triton inference request object.

Definition at line 135 of file 9.0/sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ releaseInputs() [1/2]

SharedBatchArray nvdsinferserver::TrtServerRequest::releaseInputs ( )

inline

Release the ownership of input batch buffer array.

Returns: Shared pointer to the input array.

Definition at line 152 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ releaseInputs() [2/2]

SharedBatchArray nvdsinferserver::TrtServerRequest::releaseInputs ( )

inline

Release the ownership of input batch buffer array.

Returns: Shared pointer to the input array.

Definition at line 152 of file 9.0/sources/libs/nvdsinferserver/infer_trtis_server.h.

◆ RequestOnRelease() [1/2]

static void nvdsinferserver::TrtServerRequest::RequestOnRelease	(	TRITONSERVER_InferenceRequest *	request,
		const uint32_t	flags,
		void *	userp
	)

staticprotected

The callback function to release the request instance.

Parameters

[in]	request	Pointer to the request.
[in]	flags	Flags associated with the callback.
[in]	userp	User data pointer.

◆ RequestOnRelease() [2/2]

static void nvdsinferserver::TrtServerRequest::RequestOnRelease	(	TRITONSERVER_InferenceRequest *	request,
		const uint32_t	flags,
		void *	userp
	)

staticprotected

The callback function to release the request instance.

Parameters

[in]	request	Pointer to the request.
[in]	flags	Flags associated with the callback.
[in]	userp	User data pointer.

◆ setRequestComplete() [1/2]

NvDsInferStatus nvdsinferserver::TrtServerRequest::setRequestComplete	(	TRITONSERVER_InferenceRequestReleaseFn_t	requestCompleteCb,
		void *	userPtr
	)

protected

Set the release callback function for the request.

Parameters

[in]	requestCompleteCb	The request release callback function.
[in]	userPtr	The user data pointer for the callback.

Returns: NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

◆ setRequestComplete() [2/2]

NvDsInferStatus nvdsinferserver::TrtServerRequest::setRequestComplete	(	TRITONSERVER_InferenceRequestReleaseFn_t	requestCompleteCb,
		void *	userPtr
	)

protected

Set the release callback function for the request.

Parameters

[in]	requestCompleteCb	The request release callback function.
[in]	userPtr	The user data pointer for the callback.

Returns: NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

◆ setResponseComplete() [1/2]

NvDsInferStatus nvdsinferserver::TrtServerRequest::setResponseComplete	(	ShrTritonAllocator &	allocator,
		TRITONSERVER_InferenceResponseCompleteFn_t	responseCompleteCb,
		void *	responseUserPtr
	)

protected

Set the allocator and response callback for the request.

Parameters

[in]	allocator	Pointer to the output allocator instance.
[in]	responseCompleteCb	The response callback function.
[in]	responseUserPtr	The user data pointer.

Returns: NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

◆ setResponseComplete() [2/2]

NvDsInferStatus nvdsinferserver::TrtServerRequest::setResponseComplete	(	ShrTritonAllocator &	allocator,
		TRITONSERVER_InferenceResponseCompleteFn_t	responseCompleteCb,
		void *	responseUserPtr
	)

protected

Set the allocator and response callback for the request.

Parameters

[in]	allocator	Pointer to the output allocator instance.
[in]	responseCompleteCb	The response callback function.
[in]	responseUserPtr	The user data pointer.

Returns: NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

Friends And Related Function Documentation

◆ TrtISServer

TrtISServer

friend

Definition at line 70 of file sources/libs/nvdsinferserver/infer_trtis_server.h.

The documentation for this class was generated from the following file:

sources/libs/nvdsinferserver/infer_trtis_server.h

NVIDIA DeepStream SDK API Reference

8.0 Release

Detailed Description

Public Member Functions

Protected Member Functions

Static Protected Member Functions

Friends

Constructor & Destructor Documentation

◆ TrtServerRequest() [1/2]

◆ ~TrtServerRequest() [1/2]

◆ TrtServerRequest() [2/2]

◆ ~TrtServerRequest() [2/2]

Member Function Documentation

◆ bufId() [1/2]

◆ bufId() [2/2]

◆ classParams() [1/2]

◆ classParams() [2/2]

◆ id() [1/2]

◆ id() [2/2]

◆ init() [1/2]

◆ init() [2/2]

◆ model() [1/2]

◆ model() [2/2]

◆ outputs() [1/2]

◆ outputs() [2/2]

◆ ptr() [1/2]

◆ ptr() [2/2]

◆ releaseInputs() [1/2]

◆ releaseInputs() [2/2]

◆ RequestOnRelease() [1/2]

◆ RequestOnRelease() [2/2]

◆ setRequestComplete() [1/2]

◆ setRequestComplete() [2/2]

◆ setResponseComplete() [1/2]

◆ setResponseComplete() [2/2]

Friends And Related Function Documentation

◆ TrtISServer