Classes
InputWrapper(client, model_name, config) |
This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance. |
ResourcePool(create_fn[, max_size]) |
This class provides a bounded pool of resources. |
ShmInputWrapper(client, model_name, config) |
This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance. |
TritonInOut(name, bytes, datatype, shape, ...) |
Data class for model input and output configuration. |
TritonInferenceStage(c, model_name, server_url) |
Perform inference with Triton Inference Server. |
TritonInferenceWorker(inf_queue, c, ...[, ...]) |
Inference worker class for all Triton inference server requests. |