Class LLMNode#

Inheritance Relationships#

Base Type#

Derived Types#

Class Documentation#

class LLMNode : public morpheus::llm::LLMNodeBase#

This class is used to implement functionality required in the processing of an LLM service request such as a prompt generator or LLM service client. Nodes are added to an LLMEngine which manages their execution including mapping of each node’s input/output to parent and sibling nodes.

Subclassed by morpheus::llm::PyLLMNodeBase< LLMNode >, morpheus::llm::LLMEngine

Public Functions

LLMNode()#

Construct a new LLMNode object.

~LLMNode() override#

Destroy the LLMNode object.

virtual std::shared_ptr<LLMNodeRunner> add_node(
std::string name,
user_input_mappings_t inputs,
std::shared_ptr<LLMNodeBase> node,
bool is_output = false
)#

Add child node to this node and validate user input mappings.

Parameters:

  • name – child node name

  • inputs – child node input mappings

  • node – child node object

  • is_output – true if output node

Returns:

std::shared_ptr<LLMNodeRunner>

virtual std::vector<std::string> get_input_names() const override#

Get the input names for this node.

Returns:

std::vector<std::string>

const std::vector<std::string> &get_output_node_names() const#

Get the names of output child nodes.

Returns:

const std::vector<std::string>&

size_t node_count() const#

Get number of child nodes.

Returns:

size_t

virtual Task<std::shared_ptr<LLMContext>> execute(
std::shared_ptr<LLMContext> context
) override#

Execute all child nodes and save output from output node(s) to context.

Parameters:

context – context for node’s execution

Returns:

Task<std::shared_ptr<LLMContext>>