Function TRITONSERVER_ServerOptionsAddTensorFlowVgpuMemoryLimits

Function Documentation

TRITONSERVER_Error *TRITONSERVER_ServerOptionsAddTensorFlowVgpuMemoryLimits(TRITONSERVER_ServerOptions *options, int gpu_device, int num_vgpus, uint64_t per_vgpu_memory_mbytes)

Add Tensorflow virtual GPU instances to a physical GPU.

Return

a TRITONSERVER_Error indicating success or failure.

Parameters

  • options: The server options object.

  • gpu_device: The physical GPU device id.

  • num_vgpus: The number of virtual GPUs to create on the physical GPU.

  • per_vgpu_memory_mbytes: The amount of GPU memory, in megabytes, to dedicate to each virtual GPU instance.