Class LLMTaskHandlerRunner

class LLMTaskHandlerRunner

This class wraps LLMTaskHandler and maintains how its inputs map to node outputs in the LLMEngine.

Public Functions

LLMTaskHandlerRunner(input_mappings_t inputs, std::shared_ptr<LLMTaskHandler> handler)

Construct a new LLMTaskHandlerRunner object.

Parameters
  • inputs – input mappings

  • handler – task handler object

~LLMTaskHandlerRunner()

Destroy the LLMTaskHandlerRunner object.

virtual Task<LLMTaskHandler::return_t> try_handle(std::shared_ptr<LLMContext> context)

Virtual method for implementing task. Called after execution of LLM engine nodes and outputs saved to given context.

Parameters

context – context holding outputs after execution of engine nodes

Returns

Task<return_t>

inline const input_mappings_t &input_names() const

Get input names for task handler.

Returns

std::vector<std::string>

Previous Class LLMTaskHandler
Next Class PyLLMEngine
© Copyright 2024, NVIDIA. Last updated on Apr 25, 2024.