Function TRITONSERVER_ServerOptionsSetCudaMemoryPoolByteSize

Function Documentation

TRITONSERVER_Error *TRITONSERVER_ServerOptionsSetCudaMemoryPoolByteSize(TRITONSERVER_ServerOptions *options, int gpu_device, uint64_t size)

Set the total CUDA memory byte size that the server can allocate on given GPU device in a server options.

This option will not affect the allocation conducted by the backend frameworks.

Return

a TRITONSERVER_Error indicating success or failure.

Parameters

  • options: The server options object.

  • gpu_device: The GPU device to allocate the memory pool.

  • size: The CUDA memory pool byte size.