Function TRITONSERVER_ServerInferAsync

Function Documentation

TRITONSERVER_Error *TRITONSERVER_ServerInferAsync(TRITONSERVER_Server *server, TRITONSERVER_InferenceRequest *inference_request, TRITONSERVER_InferenceTrace *trace)

Perform inference using the meta-data and inputs supplied by the ‘inference_request’.

If the function returns success, then the caller releases ownership of ‘inference_request’ and must not access it in any way after this call, until ownership is returned via the ‘request_release_fn’ callback registered in the request object with TRITONSERVER_InferenceRequestSetReleaseCallback.

The function unconditionally takes ownership of ‘trace’ and so the caller must not access it in any way after this call (except in the trace id callback) until ownership is returned via the trace’s release_fn callback.

Responses produced for this request are returned using the allocator and callback registered with the request by TRITONSERVER_InferenceRequestSetResponseCallback.

Return

a TRITONSERVER_Error indicating success or failure.

Parameters

  • server: The inference server object.

  • inference_request: The request object.

  • trace: The trace object for this request, or nullptr if no tracing.