Class LLMTaskHandler

• Defined in File llm_task_handler.hpp

Inheritance Relationships

Derived Type

• public morpheus::llm::PyLLMTaskHandler (Class PyLLMTaskHandler)

Class Documentation

class LLMTaskHandler

Base class for implementing task handlers used to process node outputs in LLMEngine.

Subclassed by morpheus::llm::PyLLMTaskHandler

Public Types

using return_t = std::optional<std::vector<std::shared_ptr<ControlMessage>>>

Public Functions

virtual ~LLMTaskHandler() = default

virtual std::vector<std::string> get_input_names() const = 0

Virtual method for implementing how task handler gets its input names.

Returns:

std::vector<std::string>

virtual Task<return_t> try_handle(

std::shared_ptr<LLMContext> context

) = 0

Virtual method for implementing execution of task. Called after execution of LLM engine nodes and outputs saved to given context.

Parameters:

context – context holding outputs after execution of engine nodes

Returns:

Task<return_t>