Class LLMNode

• Defined in File llm_node.hpp

Base Type

• public morpheus::llm::LLMNodeBase (Class LLMNodeBase)

Derived Types

• public morpheus::llm::PyLLMNodeBase< LLMNode > (Template Class PyLLMNodeBase)

• public morpheus::llm::LLMEngine (Class LLMEngine)

class LLMNode : public morpheus::llm::LLMNodeBase

This class is used to implement functionality required in the processing of an LLM service request such as a prompt generator or LLM service client. Nodes are added to an LLMEngine which manages their execution including mapping of each node’s input/output to parent and sibling nodes.

Subclassed by morpheus::llm::PyLLMNodeBase< LLMNode >, morpheus::llm::LLMEngine

Public Functions

LLMNode()

Construct a new LLMNode object.

~LLMNode() override

Destroy the LLMNode object.

virtual std::shared_ptr<LLMNodeRunner> add_node(std::string name, user_input_mappings_t inputs, std::shared_ptr<LLMNodeBase> node, bool is_output = false)

Add child node to this node and validate user input mappings.

Parameters

• name – child node name

• inputs – child node input mappings

• node – child node object

• is_output – true if output node

Returns

std::shared_ptr<LLMNodeRunner>

virtual std::vector<std::string> get_input_names() const override

Get the input names for this node.

Returns

std::vector<std::string>

const std::vector<std::string> &get_output_node_names() const

Get the names of output child nodes.

Returns

const std::vector<std::string>&

size_t node_count() const

Get number of child nodes.

Returns

size_t

virtual Task<std::shared_ptr<LLMContext>> execute(std::shared_ptr<LLMContext> context) override

Execute all child nodes and save output from output node(s) to context.

Parameters

context – context for node’s execution

Returns

Task>