Class TritonInferenceClientSession

• Defined in File triton_inference.hpp

Inheritance Relationships

Base Type

• public morpheus::IInferenceClientSession (Class IInferenceClientSession)

Class Documentation

class TritonInferenceClientSession : public morpheus::IInferenceClientSession

Public Functions

TritonInferenceClientSession(

std::shared_ptr<ITritonClient> client,

std::string model_name,

bool force_convert_inputs

)

virtual std::vector<TensorModelMapping> get_input_mappings(

std::vector<TensorModelMapping> input_map_overrides

) override

Gets the inference input mappings for Triton.

virtual std::vector<TensorModelMapping> get_output_mappings(

std::vector<TensorModelMapping> output_map_overrides

) override

Gets the inference output mappings for Triton.

virtual mrc::coroutines::Task<TensorMap> infer(

TensorMap &&inputs

) override

Invokes a single tensor inference using the constructor-provided ITritonClient.