Class LLMEngine
Defined in File llm_engine.hpp
Base Type
public morpheus::llm::LLMNode
(Class LLMNode)
-
class LLMEngine : public morpheus::llm::LLMNode
A subclass of LLMNode that acts as container for components required in processing LLM service requests. For example, a prompt generator and LLM service can each be implemented as an LLMNode and added to the engine. Input mappings are used to match node’s input to outputs of parent or sibling nodes. Task handlers can also be added to the engine to perform additional processing on the outputs stored in the LLMContext of the engine.
Public Functions
-
LLMEngine()
Construct a new LLMEngine object.
-
~LLMEngine() override
Destroy the LLMEngine object.
Add new task handler to this engine.
- Parameters
inputs – input mappings that specifies inputs to new task handler
task_handler – task handler object
Execute nodes in this engine and pass outputs to its task handlers. Must pass this a control message with a ‘llm_engine’ task containing ‘task_type and ‘task_dict’ properties required for execution of task(s).
- Parameters
input_message – input control message
- Returns
-
LLMEngine()