NVIDIA Morpheus (24.06)
(Latest Version)

Struct InferenceClientStageInterfaceProxy

struct InferenceClientStageInterfaceProxy

Interface proxy, used to insulate python bindings.

Public Static Functions

static std::shared_ptr<mrc::segment::Object<InferenceClientStage<MultiInferenceMessage, MultiResponseMessage>>> init_mm(mrc::segment::Builder &builder, const std::string &name, std::string model_name, std::string server_url, bool needs_logits, bool force_convert_inputs, std::map<std::string, std::string> input_mapping, std::map<std::string, std::string> output_mapping)

Create and initialize a MultiMessage-based InferenceClientStage, and return the result.

Parameters
  • builder – : Pipeline context object reference

  • name – : Name of a stage reference

  • model_name – : Name of the model specifies which model can handle the inference requests that are sent to Triton inference

  • server_url – : Triton server URL.

  • needs_logits – : Determines if logits are required.

  • force_convert_inputs – : Determines if inputs should be converted to the model’s input format.

  • inout_mapping – : Dictionary used to map pipeline input/output names to Triton input/output names. Use this if the Morpheus names do not match the model.

Returns

std::shared_ptr<mrc::segment::Object<InferenceClientStage<MultiInferenceMessage, MultiResponseMessage>>>

static std::shared_ptr<mrc::segment::Object<InferenceClientStage<ControlMessage, ControlMessage>>> init_cm(mrc::segment::Builder &builder, const std::string &name, std::string model_name, std::string server_url, bool needs_logits, bool force_convert_inputs, std::map<std::string, std::string> input_mapping, std::map<std::string, std::string> output_mapping)

Create and initialize a ControlMessage-based InferenceClientStage, and return the result.

Parameters
  • builder – : Pipeline context object reference

  • name – : Name of a stage reference

  • model_name – : Name of the model specifies which model can handle the inference requests that are sent to Triton inference

  • server_url – : Triton server URL.

  • needs_logits – : Determines if logits are required.

  • force_convert_inputs – : Determines if inputs should be converted to the model’s input format.

  • inout_mapping – : Dictionary used to map pipeline input/output names to Triton input/output names. Use this if the Morpheus names do not match the model.

Returns

std::shared_ptr<mrc::segment::Object<InferenceClientStage<ControlMessage, ControlMessage>>>

Previous Struct IDataTable
Next Struct InferenceMemoryFILInterfaceProxy
© Copyright 2024, NVIDIA. Last updated on Jul 8, 2024.