morpheus.stages.inference.triton_inference_stage
Classes
InputWrapper (client, model_name, config) |
This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance. |
ResourcePool (create_fn[, max_size]) |
This class provides a bounded pool of resources. |
ShmInputWrapper (client, model_name, config) |
This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance. |
TritonInOut (name, bytes, datatype, shape, ...) |
Data class for model input and output configuration. |
TritonInferenceStage (c, model_name, server_url) |
Perform inference with Triton Inference Server. |
TritonInferenceWorker (inf_queue, c, ...[, ...]) |
Inference worker class for all Triton inference server requests. |