Class TritonInferenceClientSession

(Latest Version)

Base Type

class TritonInferenceClientSession : public morpheus::IInferenceClientSession

Public Functions

TritonInferenceClientSession(std::shared_ptr<ITritonClient> client, std::string model_name)

virtual std::vector<TensorModelMapping> get_input_mappings(std::vector<TensorModelMapping> input_map_overrides) override

Gets the inference input mappings for Triton.

virtual std::vector<TensorModelMapping> get_output_mappings(std::vector<TensorModelMapping> output_map_overrides) override

Gets the inference output mappings for Triton.

virtual mrc::coroutines::Task<TensorMap> infer(TensorMap &&inputs) override

Invokes a single tensor inference using the constructor-provided ITritonClient.

Previous Class TritonInferenceClient
Next Class WriteToFileStage
© Copyright 2024, NVIDIA. Last updated on Apr 25, 2024.