Function TRTSERVER_SharedMemoryBlockGpuNew

Function Documentation

TRTSERVER_Error *TRTSERVER_SharedMemoryBlockGpuNew(TRTSERVER_SharedMemoryBlock **shared_memory_block, const char *name, const cudaIpcMemHandle_t *cuda_shm_handle, const size_t byte_size, const int device_id)

Create a new shared memory block object referencing a CUDA shared memory block residing in TRTSERVER_MEMORY_GPU type memory.

Return

a TRTSERVER_Error indicating success or failure.

Parameters

  • shared_memory_block: Returns the new shared memory block object.

  • name: A unique name for the shared memory block. This name is used in inference requests to refer to this shared memory block.

  • cuda_shm_handle: The CUDA IPC handle.

  • byte_size: The size, in bytes of the block.

  • device_id: The GPU number the CUDA shared memory region is in.