Class TritonInferenceClientSession

• Defined in File triton_inference.hpp

Base Type

• public morpheus::IInferenceClientSession (Class IInferenceClientSession)

class TritonInferenceClientSession : public morpheus::IInferenceClientSession

Public Functions

TritonInferenceClientSession(std::shared_ptr<ITritonClient> client, std::string model_name)

virtual std::vector<TensorModelMapping> get_input_mappings(std::vector<TensorModelMapping> input_map_overrides) override

Gets the inference input mappings for Triton.

virtual std::vector<TensorModelMapping> get_output_mappings(std::vector<TensorModelMapping> output_map_overrides) override

Gets the inference output mappings for Triton.

virtual mrc::coroutines::Task<TensorMap> infer(TensorMap &&inputs) override

Invokes a single tensor inference using the constructor-provided ITritonClient.