NVIDIA Morpheus (24.06)

Class TritonInferenceClientSession

Base Type

class TritonInferenceClientSession : public morpheus::IInferenceClientSession

Public Functions

TritonInferenceClientSession(std::shared_ptr<ITritonClient> client, std::string model_name, bool force_convert_inputs)

virtual std::vector<TensorModelMapping> get_input_mappings(std::vector<TensorModelMapping> input_map_overrides) override

Gets the inference input mappings for Triton.

virtual std::vector<TensorModelMapping> get_output_mappings(std::vector<TensorModelMapping> output_map_overrides) override

Gets the inference output mappings for Triton.

virtual mrc::coroutines::Task<TensorMap> infer(TensorMap &&inputs) override

Invokes a single tensor inference using the constructor-provided ITritonClient.

Previous Class TritonInferenceClient
Next Class json_dict_t
© Copyright 2024, NVIDIA. Last updated on Jul 8, 2024.