nemoguardrails.actions.llm.generation
A set of actions for generating various types of completions using an LLMs.
Module Contents
Classes
Functions
Data
API
A container objects for multiple related actions.
Return the bot message cached by the single LLM call, or None to fall back.
Returns None when the cache cannot be used — the last user-intent event
is not a UserIntent, or its cached bot intent does not match the bot
intent now being generated — in which case the caller regenerates.
Build the few-shot examples and candidate intents for the single call.
Searches the user-message index for utterances similar to text, pairs
each candidate intent with a flow (and its bot message, if any) from the
flows / bot-message indexes, and returns up to five formatted examples
plus the list of candidate user intents.
Detect the canonical form (user intent) for the given user message.
Covers both the embeddings-only lookup and the LLM-based canonical-form
generation. Always returns a single UserIntent event.
Evict a pending single-call streaming handoff for a bypassed bot message.
When generate_bot_message short-circuits to a predefined message or a
$context_var it never consumes the single-call cache, so a handoff
registered while streaming the (now unused) cached bot message would leak.
This releases it. Safe to call when there is no pending handoff.
Generate and package a general bot turn when there are no user messages.
Handles the passthrough (with or without a passthrough fn) and the
non-passthrough general paths, then packages the result into the
BotMessage/BotToolCalls/BotThinking events and context updates that
generate_user_intent returns in this mode.
Heuristic to extract user message examples from a flow.
Make a single general-response LLM call and parse its output.
This is the duplicated core of the four Task.GENERAL call sites: the
general user-intent fallback, the passthrough completion, the single-call
general branch and the passthrough bot-message branch. Prompt rendering,
chunk retrieval and any output stripping stay at the call sites; only the
LLMCallInfo set, the llm_call and the parse are shared here. The
stop, stream_during_call, llm_call_task and parse_task
parameters preserve the per-site differences (e.g. the passthrough
bot-message branch reports GENERATE_BOT_MESSAGE but parses as
GENERAL).
Helper to extract the general instruction.
Helper to extract only the two turns from the sample conversation.
This is needed to be included to “seed” the conversation so that the model can follow the format more easily.
Initializes the index of bot messages.
Initializes the index of flows.
Initializes the index of user messages.
Process the provided flows to extract the user utterance examples.
Render a string using the provided context information.
Parameters:
The string template to render.
The context for rendering.
Returns: str
The rendered string.
Search the index of flows.
Generate a bot message based on the desired bot intent.
Generate all three main Guardrails phases with a single LLM call. The three phases are: user canonical from (user intent), next flow steps (i.e. bot canonical form) and bot message.
Generate the next step in the current conversation flow.
Currently, only generates a next step after a user intent.
Generate the canonical form for what the user said i.e. user intent.
Generate a value in the context of the conversation.
Parameters:
The instructions to generate the value.
The full stream of events so far.
The name of the variable to generate. If not specified, it will use the action_result_key as the name of the variable.
Optional[ LLMModel ] — Custom llm model to generate_value
The bot intent/message events computed by the single-call generation.
Carried on the additional_info of the UserIntent event so that the
later generate_next_steps and generate_bot_message phases can reuse
the results of the single LLM call instead of calling the LLM again.
Encodes the <<STREAMING[uid]>> handoff used by the single-call path.
The single-call streaming path generates the bot message on an inner
StreamingHandler, registers it here and leaves a marker on the cached bot
message. The later bot-message phase parses the marker, takes the handler and
pipes it to the main handler. Each handler is removed when taken, so handlers
do not accumulate for the module lifetime.
Return the handler uid encoded in text, or None if not a marker.
Register an inner handler and return the marker text for it.
Remove and return the handler for uid, or None if already consumed.
Build the output events for a bot turn: an optional BotThinking then the final event.
Pops any reasoning trace into a BotThinking event placed before
final_event. When context_updates is provided, the reasoning trace is
also recorded as bot_thinking (the generated bot-message path does this;
the single-call cached path does not).
Return a SpecOp’s spec as a dict.
SpecOp.spec is Union[Spec, dict]; callers that need dict-only fields
(such as elements) use this to normalize it.
Return the (prefix, suffix) streaming pattern for the bot message.
The two call sites differ: generate_bot_message treats both verbose_v1
and bot_message as the verbose pattern, while
generate_intent_steps_message treats only verbose_v1 that way;
include_bot_message_parser selects which rule applies.
Split a passthrough fn result into (text, passthrough_output).
A tuple/list result carries an explicit passthrough output as its second element; any other return value is treated as the text output alone.
Build the single-call cache carried on the UserIntent event.
Returns the plain nested additional_info dict (not a dataclass) so the
event serializes byte-identically as it passes between actions.
Clean an utterance by performing the following operations:
- replacing “\n” with ” ”.
Args: utterance (str): The utterance to clean.
Returns: str: The cleaned utterance.
Read the single-call cache back from a UserIntent event.
Mirrors the existing direct dict access (raising KeyError when the
payload is absent); guarding that is a separate follow-up.