nvdsinferserver::TrtServerRequest Class Reference

Detailed Description

Wrapper class for Triton inference request.

Definition at line 68 of file infer_trtis_server.h.

Public Member Functions
	~TrtServerRequest ()
	Destructor. More...
TRITONSERVER_InferenceRequest *	ptr ()
	Get the pointer to the Triton inference request object. More...
const std::string &	model () const
	Get the model name. More...
uint64_t	id () const
	Get the request ID. More...
uint64_t	bufId () const
	Get the input buffer ID associated with the request. More...
SharedBatchArray	releaseInputs ()
	Release the ownership of input batch buffer array. More...
const std::vector< std::string > &	outputs () const
	Get the list of requested output layer names. More...
const std::map< std::string, TritonClassParams > &	classParams () const
	Get the Triton classification parameters list (tensor name : classification parameters). More...

Protected Member Functions
	TrtServerRequest (TrtServerPtr server)
	Constructor. More...
NvDsInferStatus	init (const std::string &model, int64_t version, SharedBatchArray &inputs, const std::vector< std::string > &outputs, uint64_t reqId, const std::vector< TritonClassParams > &clasList)
	Create a new Triton inference request with the specified inputs and parameters. More...
NvDsInferStatus	setRequestComplete (TRITONSERVER_InferenceRequestReleaseFn_t requestCompleteCb, void *userPtr)
	Set the release callback function for the request. More...
NvDsInferStatus	setResponseComplete (ShrTritonAllocator &allocator, TRITONSERVER_InferenceResponseCompleteFn_t responseCompleteCb, void *responseUserPtr)
	Set the allocator and response callback for the request. More...

Static Protected Member Functions
static void	RequestOnRelease (TRITONSERVER_InferenceRequest *request, const uint32_t flags, void *userp)
	The callback function to release the request instance. More...

Friends
class	TrtISServer

Constructor & Destructor Documentation

TrtServerRequest()

nvdsinferserver::TrtServerRequest::TrtServerRequest ( TrtServerPtr  server )

protected

Constructor.

Save the server instance pointer and register the Triton request deletion function.

Parameters

server

~TrtServerRequest()

nvdsinferserver::TrtServerRequest::~TrtServerRequest

(

)

Destructor.

Releases the Triton inference request instance.

Member Function Documentation

bufId()

uint64_t nvdsinferserver::TrtServerRequest::bufId ( ) const

inline

Get the input buffer ID associated with the request.

Definition at line 147 of file infer_trtis_server.h.

classParams()

const std::map<std::string, TritonClassParams>& nvdsinferserver::TrtServerRequest::classParams ( ) const

inline

Get the Triton classification parameters list (tensor name : classification parameters).

Definition at line 161 of file infer_trtis_server.h.

id()

uint64_t nvdsinferserver::TrtServerRequest::id ( ) const

inline

Get the request ID.

Definition at line 143 of file infer_trtis_server.h.

init()

NvDsInferStatus nvdsinferserver::TrtServerRequest::init ( const std::string &  model, int64_t  version, SharedBatchArray &  inputs, const std::vector< std::string > &  outputs, uint64_t  reqId, const std::vector< TritonClassParams > &  clasList  )

protected

Create a new Triton inference request with the specified inputs and parameters.

Parameters

[in]	model	Model name.
[in]	version	Model version.
[in]	inputs	Array of input batch buffers.
[in]	outputs	List of names of required output tensors.
[in]	reqId	ID of this request.
[in]	clasList	Triton classification parameters, if any.

Returns

NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

model()

const std::string& nvdsinferserver::TrtServerRequest::model ( ) const

inline

Get the model name.

Definition at line 139 of file infer_trtis_server.h.

outputs()

const std::vector<std::string>& nvdsinferserver::TrtServerRequest::outputs ( ) const

inline

Get the list of requested output layer names.

Definition at line 156 of file infer_trtis_server.h.

ptr()

TRITONSERVER_InferenceRequest* nvdsinferserver::TrtServerRequest::ptr ( )

inline

Get the pointer to the Triton inference request object.

Definition at line 135 of file infer_trtis_server.h.

releaseInputs()

SharedBatchArray nvdsinferserver::TrtServerRequest::releaseInputs ( )

inline

Release the ownership of input batch buffer array.

Returns

Shared pointer to the input array.

Definition at line 152 of file infer_trtis_server.h.

RequestOnRelease()

static void nvdsinferserver::TrtServerRequest::RequestOnRelease ( TRITONSERVER_InferenceRequest *  request, const uint32_t  flags, void *  userp  )

staticprotected

The callback function to release the request instance.

Parameters

[in]	request	Pointer to the request.
[in]	flags	Flags associated with the callback.
[in]	userp	User data pointer.

setRequestComplete()

NvDsInferStatus nvdsinferserver::TrtServerRequest::setRequestComplete ( TRITONSERVER_InferenceRequestReleaseFn_t  requestCompleteCb, void *  userPtr  )

protected

Set the release callback function for the request.

Parameters

[in]	requestCompleteCb	The request release callback function.
[in]	userPtr	The user data pointer for the callback.

Returns

NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

setResponseComplete()

NvDsInferStatus nvdsinferserver::TrtServerRequest::setResponseComplete ( ShrTritonAllocator &  allocator, TRITONSERVER_InferenceResponseCompleteFn_t  responseCompleteCb, void *  responseUserPtr  )

protected

Set the allocator and response callback for the request.

Parameters

[in]	allocator	Pointer to the output allocator instance.
[in]	responseCompleteCb	The response callback function.
[in]	responseUserPtr	The user data pointer.

Returns

NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.

Friends And Related Function Documentation

TrtISServer

friend class TrtISServer

friend

Definition at line 70 of file infer_trtis_server.h.

---

The documentation for this class was generated from the following file:

• infer_trtis_server.h