Function TRITONSERVER_InferenceRequestSetResponseCallback

Function Documentation

TRITONSERVER_Error *TRITONSERVER_InferenceRequestSetResponseCallback(TRITONSERVER_InferenceRequest *inference_request, TRITONSERVER_ResponseAllocator *response_allocator, void *response_allocator_userp, TRITONSERVER_InferenceResponseCompleteFn_t response_fn, void *response_userp)

Set the allocator and response callback for an inference request.

The allocator is used to allocate buffers for any output tensors included in responses that are produced for this request. The response callback is called to return response objects representing responses produced for this request.

Return

a TRITONSERVER_Error indicating success or failure.

Parameters

  • inference_request: The request object.

  • response_allocator: The TRITONSERVER_ResponseAllocator to use to allocate buffers to hold inference results.

  • response_allocator_userp: User-provided pointer that is delivered to the response allocator’s start and allocation functions.

  • response_fn: The function called to deliver an inference response for this request.

  • response_userp: User-provided pointer that is delivered to the ‘response_fn’ callback.