morpheus.stages.inference.triton_inference_stage

Classes

`InputWrapper`(client, model_name, config)	This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance.
`ResourcePool`(create_fn[, max_size])	This class provides a bounded pool of resources.
`ShmInputWrapper`(client, model_name, config)	This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance.
`TritonInOut`(name, bytes, datatype, shape, ...)	Data class for model input and output configuration.
`TritonInferenceAE`(inf_queue, c, model_name, ...)	This class extends `TritonInference` to deal with inference processing specific to the AutoEncoder.
`TritonInferenceFIL`(inf_queue, c, model_name, ...)	This class extends `TritonInference` to deal with scenario-specific FIL models inference requests like building response.
`TritonInferenceNLP`(inf_queue, c, model_name, ...)	This class extends TritonInference to deal with scenario-specific NLP models inference requests like building response.
`TritonInferenceStage`(c, model_name, server_url)	Perform inference with Triton Inference Server.