morpheus.stages.inference.triton_inference_stage

Classes

InputWrapper(client, model_name, config) This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance.
ResourcePool(create_fn[, max_size]) This class provides a bounded pool of resources.
ShmInputWrapper(client, model_name, config) This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance.
TritonInOut(name, bytes, datatype, shape, ...) Data class for model input and output configuration.
TritonInferenceAE(inf_queue, c, model_name, ...) This class extends TritonInference to deal with inference processing specific to the AutoEncoder.
TritonInferenceFIL(inf_queue, c, model_name, ...) This class extends TritonInference to deal with scenario-specific FIL models inference requests like building response.
TritonInferenceNLP(inf_queue, c, model_name, ...) This class extends TritonInference to deal with scenario-specific NLP models inference requests like building response.
TritonInferenceStage(c, model_name, server_url) Perform inference with Triton Inference Server.
Previous morpheus.stages.inference.pytorch_inference_stage.PyTorchInferenceStage
Next morpheus.stages.inference.triton_inference_stage.InputWrapper
© Copyright 2023, NVIDIA. Last updated on Feb 2, 2024.