Function TRTSERVER_ServerInferAsync¶
Defined in File trtserver.h
Function Documentation¶
-
TRTSERVER_Error *
TRTSERVER_ServerInferAsync
(TRTSERVER_Server *server, TRTSERVER_TraceManager *trace_manager, TRTSERVER_InferenceRequestProvider *request_provider, TRTSERVER_ResponseAllocator *response_allocator, void *response_allocator_userp, TRTSERVER_InferenceCompleteFn_t complete_fn, void *complete_userp)¶ Perform inference using the meta-data and inputs supplied by the request provider.
The caller retains ownership of ‘request_provider’ but may release it by calling TRTSERVER_InferenceRequestProviderDelete once this function returns.
- Return
a TRTSERVER_Error indicating success or failure.
- Parameters
server
: The inference server object.trace_manager
: The trace manager object for this request, or nullptr if no tracing.request_provider
: The request provider for the request.response_allocator
: The TRTSERVER_ResponseAllocator to use to allocate buffers to hold inference results.response_allocator_userp
: User-provided pointer that is delivered to the response allocator’s allocation function.complete_fn
: The function called when the inference completes.complete_userp
: User-provided pointer that is delivered to the completion function.