Generation Options Reference

Generation options let you control which rails run, which parameters the main large language model (LLM) receives, and which context and logging data the NVIDIA NeMo Guardrails library returns.

To use generation options, provide the options keyword argument to the generate() or generate_async() methods:

1
messages = [{
2
    "role": "user",
3
    "content": "..."
4
}]
5
rails.generate(messages=messages, options={...})

Engine Support

Both the LLMRails and IORails engines accept generation options, and both return a structured GenerationResponse object when you pass options. Because IORails does not run the Colang runtime, the options that read or write Colang state are unavailable on it, and a few GenerationResponse fields are built differently. Each section below documents the shared behavior first and then the IORails difference. For the complete field-by-field comparison, refer to IORails and LLMRails Differences.

1
# No options: an OpenAI-style message dictionary.
2
message = rails.generate(messages=messages)
3
print(message["content"])
4
5
# With options: a GenerationResponse.
6
result = rails.generate(messages=messages, options={"log": {"llm_calls": True}})
7
print(result.response[0]["content"])
8
print(result.reasoning_content)

Disabling Rails

Use the rails generation option to choose which rail categories to apply. The supported categories are input, output, dialog, retrieval, tool_input, and tool_output. By default, all categories are enabled.

IORails runs input, output, and tool rails only. It ignores the dialog and retrieval categories rather than raising, because a configuration that declares dialog or retrieval rails routes to LLMRails in the first place.

1
res = rails.generate(messages=messages)

runs the same rails as:

1
res = rails.generate(messages=messages, options={
2
    "rails": ["input", "output", "dialog", "retrieval", "tool_input", "tool_output"]
3
})

The two calls return different types because passing options selects the structured GenerationResponse return.

The list form is exhaustive: every category you leave out of the list is disabled. To disable one category and keep the rest, use the dictionary form instead.

1
res = rails.generate(messages=messages, options={
2
    "rails": {"dialog": False}
3
})

Input Rails Only

To check a user’s input with only the input rails from a guardrails configuration, disable all other rail categories:

1
res = rails.generate(messages=[{
2
    "role": "user",
3
    "content": "Some user input."
4
}], options={
5
    "rails": ["input"]
6
})

The response will be the same string if the input was allowed “as is”:

1
{
2
  "role": "assistant",
3
  "content": "Some user input."
4
}

If some of the rails alter the input, for example, to mask sensitive information, then the returned value is the altered input.

1
{
2
  "role": "assistant",
3
  "content": "Some altered user input."
4
}

If the input was blocked, you will get the predefined response bot refuse to respond (by default “I’m sorry, I can’t respond to that”).

1
{
2
  "role": "assistant",
3
  "content": "I'm sorry, I can't respond to that."
4
}

For more details on what rails was triggered, use the log.activated_rails generation option.

Input and Output Rails Only

If you want to check both the user input and an output that was generated outside of the guardrails configuration, you must disable the dialog rails and the retrieval rails, and provide a bot message as well when making the call:

1
res = rails.generate(messages=[{
2
    "role": "user",
3
    "content": "Some user input."
4
}, {
5
    "role": "assistant",
6
    "content": "Some bot output."
7
}], options={
8
    "rails": ["input", "output"]
9
})

The response will be the exact bot message provided, if allowed, an altered version if an output rail decides to change it, for example, to remove sensitive information, or the predefined message for bot refuse to respond, if the message was blocked.

For receive details on what rails are triggered, use the log.activated_rails generation option.

Worked Example: Compare All Rails to Input and Output Rails

The topical rails tutorial uses an ABC bot configuration with input, dialog, generation, and output rails. When all rails are enabled, a simple greeting can activate several rails and trigger multiple LLM calls:

{'type': 'input', 'name': 'self check input'}
{'type': 'dialog', 'name': 'generate user intent'}
{'type': 'dialog', 'name': 'generate next step'}
{'type': 'generation', 'name': 'generate bot message'}
{'type': 'output', 'name': 'self check output'}
{'type': 'output', 'name': 'check blocked terms'}

The explain() method can show the corresponding LLM call count:

1
info = rails.explain()
2
info.print_llm_calls_summary()

Summary: 5 LLM call(s) took 3.54 seconds and used 1621 tokens.

If you only need to validate an already-generated assistant message, provide both the user and assistant messages and set options={"rails": ["input", "output"]}. This skips dialog, retrieval, and generation rails while still applying the configured input and output checks.

For validation-only use cases, prefer the check() and check_async() APIs, which run input and output rails without invoking full generation.

Output Rails Only

To apply output rails exclusively to an LLM response, disable the input rails and provide an empty input.

1
res = rails.generate(messages=[{
2
    "role": "user",
3
    "content": ""
4
}, {
5
    "role": "assistant",
6
    "content": "Some bot output."
7
}], options={
8
    "rails": ["output"]
9
})

Detailed Logging Information

You can obtain detailed information about what happened under the hood during the generation process by setting the log generation option. This option has four different inner-options:

• activated_rails: Include detailed information about the rails that were activated during generation.
• llm_calls: Include the prompt, completion, timing, token usage, request ID, model, and provider information available for each LLM call.
• internal_events: Include the array of internal generated events.
• colang_history: Include the history of the conversation in Colang format.

1
res = rails.generate(messages=messages, options={
2
    "log": {
3
        "activated_rails": True,
4
        "llm_calls": True,
5
        "internal_events": True,
6
        "colang_history": True
7
    }
8
})

{
  "response": [...],
  "log": {
    "activated_rails": {
      ...
    },
    "stats": {...},
    "llm_calls": [...],
    "internal_events": [...],
    "colang_history": "..."
  }
}

When using the Python API, the log is an object that also has a print_summary method. When called, it will print a simplified version of the log information. Below is a sample output.

1
res.log.print_summary()

1
# General stats
2
3
- Total time: 2.85s
4
  - [0.56s][19.64%]: INPUT Rails
5
  - [1.40s][49.02%]: DIALOG Rails
6
  - [0.58s][20.22%]: GENERATION Rails
7
  - [0.31s][10.98%]: OUTPUT Rails
8
- 5 LLM calls, 2.74s total duration, 1641 total prompt tokens, 103 total completion tokens, 1744 total tokens.
9
10
# Detailed stats
11
12
- [0.56s] INPUT (self check input): 1 actions (self_check_input), 1 llm calls [0.56s]
13
- [0.43s] DIALOG (generate user intent): 1 actions (generate_user_intent), 1 llm calls [0.43s]
14
- [0.96s] DIALOG (generate next step): 1 actions (generate_next_step), 1 llm calls [0.95s]
15
- [0.58s] GENERATION (generate bot message): 2 actions (retrieve_relevant_chunks, generate_bot_message), 1 llm calls [0.49s]
16
- [0.31s] OUTPUT (self check output): 1 actions (self_check_output), 1 llm calls [0.31s]

Log Options on IORails

IORails supports activated_rails and llm_calls, and it includes stats whenever you request either one, exactly as LLMRails does.

internal_events and colang_history describe Colang runtime state, so IORails raises NotImplementedError when you set either to True rather than returning an empty value. The following example requests the supported log fields:

1
res = rails.generate(messages=messages, options={
2
    "log": {
3
        "activated_rails": True,
4
        "llm_calls": True
5
    }
6
})

IORails synthesizes the log from the record each rail leaves behind, so the structure matches LLMRails but the contents reflect the way IORails executes rails:

• Each activated rail carries exactly one ExecutedAction, holding the rail’s structured verdict in return_value and up to one LLM or API call. IORails runs one model-backed check per rail instead of a Colang action chain.
• activated_rails[].decisions is always an empty list, because decisions are a Colang construct.
• The main model call appears as a rail with type generation, name generation, and action name generate_bot_message. On LLMRails the equivalent entries come from the Colang dialog rails, such as generate user intent.
• llm_calls[].prompt is the conversation serialized as "<role>: <content>" lines. The content matches what LLMRails logs, but the formatting differs.
• Each call record includes timing and the request ID, token usage, model name, and provider name available from the provider. The stats object aggregates the available values the same way as LLMRails.
• A request stopped by a rail still returns the log for the rails that ran before processing stopped.

Output Variables

Some rails can store additional information in Colang 1.0 Language Syntax: Variables. You can return the content of these variables by setting the output_vars generation option to the list of names for all the variables that you are interested in. If you want to return the complete context (this will also include some predefined variables), you can set output_vars to True.

1
rails.generate(messages=messages, options={
2
    "output_vars": ["some_input_rail_score", "some_output_rail_score"]
3
})

You can find the returned data in the output_data key of the response:

{
  "response": [...],
  "output_data": {
    "some_input_rail_score": 0.7,
    "some_output_rail_score": 0.8
  }
}

Additional LLM Parameters

To supply additional parameters to the LLM call during final message generation, utilize the llm_params option. The following example demonstrates how to apply a lower value for temperature:

1
rails.generate(messages=messages, options={
2
    "llm_params": {
3
        "temperature": 0.2
4
    }
5
})

The available parameters are determined by the specific LLM engine in use. The NeMo Guardrails library transmits values defined in the options parameter without modification.

Both engines apply llm_params to the main model call only, not to the models that back the rails. IORails also accepts llm_params in stream_async(), and it is the supported way to pass tool definitions to the main model.

Additional LLM Output

You can receive additional output from the LLM generation by setting llm_output to True through the options parameter.

1
rails.generate(messages=messages, options={
2
    "llm_output": True
3
})

IORails and LLMRails Differences

This section collects the differences between the two engines in one place. For capability differences outside generation options, refer to Engine Feature Support.

Method Signature

IORails.generate() and IORails.generate_async() take the same first three parameters as LLMRails.

1
def generate(
2
    self,
3
    prompt: Optional[str] = None,
4
    messages: Optional[LLMMessages] = None,
5
    options: Optional[Union[dict, GenerationOptions]] = None,
6
    **kwargs,
7
) -> Union[LLMMessage, GenerationResponse]:

Three argument-handling details differ from LLMRails:

• When you pass both prompt and messages, IORails uses messages and ignores prompt. LLMRails raises ValueError.
• When you pass a prompt and options, LLMRails sets GenerationResponse.response to a plain string. IORails always sets it to a one-element list holding the assistant message.
• An empty options={} returns a GenerationResponse on IORails and raises TypeError on LLMRails. Pass options=GenerationOptions() for the structured return with default settings on either engine.

Passing a message list as the first positional argument raises TypeError on IORails, because that slot is prompt. Pass the list with the messages= keyword.

1
# Raises TypeError on IORails.
2
rails.generate([{"role": "user", "content": "Hello"}])
3
4
# Correct.
5
rails.generate(messages=[{"role": "user", "content": "Hello"}])

IORails does not support the state and streaming_handler keyword arguments that LLMRails accepts. state raises ValueError, because IORails is a stateless input and output rails engine with nothing to persist or resume. streaming_handler is accepted and ignored. Use stream_async() to stream from IORails.

GenerationOptions Support

stream_async() accepts options too, but reads only a subset of it and never yields a GenerationResponse. Refer to Generation Options and Streaming.

Legend: ✓ supported · ✗ not supported (raises) · ◐ accepted but not populated · N/A not applicable to the engine.

The following table compares generation option support for generate() and generate_async():

IORails raises for Colang-dependent options that it cannot honor, including output_vars, log.internal_events, and log.colang_history. It accepts and ignores llm_output, rails.dialog, and rails.retrieval.

state is a generate() keyword argument rather than a GenerationOptions field. enforce is commented out in the source and is not a live field on either engine.

GenerationLogOptions Support

The following table compares generation log field support:

GenerationLog.stats is not a GenerationLogOptions field. Both engines populate it whenever you request either activated_rails or llm_calls.

For the IORails log structure and how it differs from the Colang-derived log, refer to Log Options on IORails.

GenerationResponse Field Support

stream_async() never yields or returns a GenerationResponse on either engine. The following table compares the fields returned when you pass options to generate() or generate_async():

Reasoning bypasses the output rails on both engines, so output rails check the final answer rather than the reasoning trace.

Requests Stopped by Rails

When a rail blocks content, LLMRails returns the configured refusal message in response. IORails returns its fixed refusal message for a policy block and an internal-error message for a rail execution failure. In either IORails case, tool_calls, reasoning_content, and llm_metadata are None. To inspect records through the rail that stopped processing, request log.activated_rails. That rail has stop set to True, and its action’s return_value["failed"] value distinguishes a failure from a policy block.

Limitations

• Only supported for the generate/generate_async methods (not for generate_events/generate_events_async).
• On LLMRails, specifying which individual rails of a particular type to activate is not yet supported. IORails supports it: pass a list of rail names to rails.input, rails.output, rails.tool_input, or rails.tool_output.
• On LLMRails with a Colang 2.x configuration, output_vars, log, and llm_output raise ValueError.