Class PyLLMTaskHandler

Base Type

class PyLLMTaskHandler : public morpheus::llm::LLMTaskHandler

Public Functions

~PyLLMTaskHandler() override

virtual std::vector<std::string> get_input_names() const override

Virtual method for implementing how task handler gets its input names.

Returns

std::vector<std::string>

virtual Task<LLMTaskHandler::return_t> try_handle(std::shared_ptr<LLMContext> context) override

Virtual method for implementing execution of task. Called after execution of LLM engine nodes and outputs saved to given context.

Parameters

context – context holding outputs after execution of engine nodes

Returns

Task<return_t>

Previous Template Class PyLLMNodeBase
Next Class Loader
© Copyright 2023, NVIDIA. Last updated on Feb 2, 2024.