morpheus.llm.nodes.rag_node.RAGNode

Bases: morpheus._lib.llm.LLMNode

Performs Retrieval Augmented Generation (RAG).

Parameters

prompt

The prompt template string to populate with input values.

template_format

The format of the template string. Must be one of: f-string, jinja.

vdb_service

The VectorDB service to use for retrieval.

embedding

An asynchronous function to use to determine the embeddings to search the vdb_service for. If None, upstream nodes must provide the embeddings.

llm_client

The LLM client to use for generation.

Methods

add_node(self: morpheus._lib.llm.LLMNode, name: str, *, inputs: object = None, node: morpheus._lib.llm.LLMNodeBase, is_output: bool = False) → morpheus._lib.llm.LLMNodeRunner

Add an LLMNode to the current node.

Parameters

name

The name of the node to add

inputs

List of input mappings to use for the node, in the form of [(external_name, internal_name), ...] If unspecified the node’s input_names will be used.

node

The node to add

is_output

Indicates if the node is an output node, by default False

execute(self: morpheus._lib.llm.LLMNodeBase, context: morpheus._lib.llm.LLMContext) → Awaitable[morpheus._lib.llm.LLMContext]

Execute the current node with the given context instance.

All inputs for the given node should be fetched from the context, typically by calling either context.get_inputs to fetch all inputs as a dict, or context.get_input to fetch a specific input.

Similarly the output of the node is written to the context using context.set_output.

Parameters

context : morpheus._lib.llm.LLMContext

Context instance to use for the execution

get_input_names(self: morpheus._lib.llm.LLMNodeBase) → List[str]

Get the input names for the node.

Returns

list[str]

The input names for the node