Typedef TRITONSERVER_ResponseAllocatorAllocFn_t

Typedef Documentation

typedef TRITONSERVER_Error *(*TRITONSERVER_ResponseAllocatorAllocFn_t)(TRITONSERVER_ResponseAllocator *allocator, const char *tensor_name, size_t byte_size, TRITONSERVER_MemoryType memory_type, int64_t memory_type_id, void *userp, void **buffer, void **buffer_userp, TRITONSERVER_MemoryType *actual_memory_type, int64_t *actual_memory_type_id)

TRITONSERVER_ResponseAllocator.

Object representing a memory allocator for output tensors in an inference response.Type for allocation function that allocates a buffer to hold an output tensor.

Return

a TRITONSERVER_Error object if a failure occurs while attempting an allocation. If an error is returned all other return values will be ignored.

Parameters

  • allocator: The allocator that is provided in the call to TRITONSERVER_InferenceRequestSetResponseCallback.

  • tensor_name: The name of the output tensor to allocate for.

  • byte_size: The size of the buffer to allocate.

  • memory_type: The type of memory that the caller prefers for the buffer allocation.

  • memory_type_id: The ID of the memory that the caller prefers for the buffer allocation.

  • userp: The user data pointer that is provided as ‘response_allocator_userp’ in the call to TRITONSERVER_InferenceRequestSetResponseCallback.

  • buffer: Returns a pointer to the allocated memory.

  • buffer_userp: Returns a user-specified value to associate with the buffer, or nullptr if no user-specified value should be associated with the buffer. This value will be provided in the call to TRITONSERVER_ResponseAllocatorReleaseFn_t when the buffer is released and will also be returned by TRITONSERVER_InferenceResponseOutput.

  • actual_memory_type: Returns the type of memory where the allocation resides. May be different than the type of memory requested by ‘memory_type’.

  • actual_memory_type_id: Returns the ID of the memory where the allocation resides. May be different than the ID of the memory requested by ‘memory_type_id’.