Function TRTSERVER_ServerOptionsSetCudaMemoryPoolByteSize

• Defined in File trtserver.h

Function Documentation

TRTSERVER_Error *TRTSERVER_ServerOptionsSetCudaMemoryPoolByteSize(TRTSERVER_ServerOptions *options, int gpu_device, uint64_t size)

Set the total CUDA memory byte size that the server can allocate on given GPU device in a server options.

This option will not affect the allocation conducted by the backend frameworks.

Return

a TRTSERVER_Error indicating success or failure.

Parameters

• options: The server options object.

• gpu_device: The GPU device to allocate the memory pool.

• size: The CUDA memory pool byte size.