Class SharedMemoryControlContext¶
Defined in File request.h
Class Documentation¶
-
class
SharedMemoryControlContext
¶ A SharedMemoryControlContext object is used to control the registration / unregistration and get the status of shared memory regions on the inference server.
Once created, a SharedMemoryControlContext object can be used repeatedly.
A SharedMemoryControlContext object can use either HTTP protocol or GRPC protocol depending on the Create function (SharedMemoryControlHttpContext::Create or SharedMemoryControlGrpcContext::Create). For example:
std::unique_ptr<SharedMemoryControlContext> ctx; SharedMemoryControlGrpcContext::Create(&ctx, "localhost:8000"); std::string name = "shared_memory"; std::string shm_key = "/input"; ctx->RegisterSharedMemory(name, shm_key, 0, 128); ... cudaIpcMemHandle_t cuda_shm_handle; ctx->RegisterCudaSharedMemory(name, cuda_shm_handle, 128); ... ctx->UnregisterSharedMemory(name); ... ctx->UnregisterAllSharedMemory(); ... SharedMemoryStatus status; ctx->GetSharedMemoryStatus(&status); ...
Public Functions
-
virtual
~SharedMemoryControlContext
() = 0¶
-
virtual Error
RegisterSharedMemory
(const std::string &name, const std::string &shm_key, size_t offset, size_t byte_size) = 0¶ Register a system shared memory region on the inference server.
If the shared memory region is already registered, it will return error ‘TRTSERVER_ERROR_ALEADY_EXISTS’.
- Return
Error object indicating success or failure.
- Parameters
name
: The user-given name for the shared memory region to be registered.shm_key
: The unique name of the location in shared memory being registered.offset
: The offset into the shared memory region.byte_size
: The size, in bytes of the shared memory region.
-
virtual Error
RegisterCudaSharedMemory
(const std::string &name, const cudaIpcMemHandle_t &cuda_shm_handle, size_t byte_size, int device_id) = 0¶ Register a CUDA shared memory region on the inference server.
If the shared memory region is already registered, it will return error ‘TRTSERVER_ERROR_ALEADY_EXISTS’.
- Return
Error object indicating success or failure.
- Parameters
name
: The user-given name for the shared memory region to be registered.cuda_shm_handle
: The CUDA memory handle representing the location in shared memory being registered.byte_size
: The size, in bytes of the shared memory region.device
: id The GPU device ID the shared memory region is in.
-
virtual Error
UnregisterSharedMemory
(const std::string &name) = 0¶ Unregister a registered shared memory region on the inference server.
If the shared memory region is not registered, do nothing and return success.
- Return
Error object indicating success or failure.
- Parameters
name
: The user-given name for the shared memory region to be unregistered.
-
virtual Error
UnregisterAllSharedMemory
() = 0¶ Unregisters all registered shared memory regions on the inference server.
- Return
Error object indicating success or failure.
-
virtual Error
GetSharedMemoryStatus
(SharedMemoryStatus *status) = 0¶ Get the list of all active/registered shared memory regions on the inference server.
- Return
Error object indicating success or failure.
- Parameters
status
: Returns the active shared memory regions.
-
virtual