morpheus.stages.inference.triton_inference_stage
NVIDIA Docs Hub  NVIDIA Morpheus  morpheus.stages.inference.triton_inference_stage

Classes

InputWrapper(client, model_name, config)

This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance.

ResourcePool(create_fn[, max_size])

This class provides a bounded pool of resources.

ShmInputWrapper(client, model_name, config)

This class is a wrapper around a CUDA shared memory object shared between this process and a Triton server instance.

TritonInOut(name, bytes, datatype, shape, ...)

Data class for model input and output configuration.

TritonInferenceAE(inf_queue, c, model_name, ...)

This class extends TritonInference to deal with inference processing specific to the AutoEncoder.

TritonInferenceFIL(inf_queue, c, model_name, ...)

This class extends TritonInference to deal with scenario-specific FIL models inference requests like building response.

TritonInferenceNLP(inf_queue, c, model_name, ...)

This class extends TritonInference to deal with scenario-specific NLP models inference requests like building response.

TritonInferenceStage(c, model_name, server_url)

Perform inference with Triton Inference Server.
© Copyright 2023, NVIDIA. Last updated on Apr 11, 2023.