Configure PluginsNeMo Guardrails

NeMo Guardrails Plugin

View as Markdown

Use the NeMo Guardrails plugin when you want first-party Guardrails policy around managed NeMo Relay LLM and tool execution through the shared plugin system.

The built-in plugin component has kind nemo_guardrails and is available as a first-party NeMo Relay plugin.

The plugin supports these backend modes:

  • remote
    • Calls a Guardrails service over HTTP(S), including streaming over the same remote contract.
  • local
    • Calls nemoguardrails through a local python3 worker subprocess instead of a separate Guardrails service.

Use This Plugin When

Start here when you need to:

  • Apply Guardrails input and output checks around managed llm.execute(...) calls.
  • Apply Guardrails policy around managed tool execution.
  • Configure Guardrails behavior through the same plugin config surface used by other first-party NeMo Relay components.
  • Keep Guardrails policy authoring in Guardrails-native config while NeMo Relay owns when those checks run around managed execution.

Current Scope

The built-in plugin currently exposes two user-facing modes:

  • remote for Guardrails-service integration over HTTP(S)
  • local for nemoguardrails integration through a local Python worker

Both modes support managed LLM input and output. The current mode-specific differences are:

  • remote supports request_defaults pass-through but does not support managed tool_input
  • local supports managed tool_input and broader LLM codec coverage, but it does not support request_defaults

The local backend requires a python3 >= 3.11 executable that can import nemoguardrails==0.22.0. It does not embed Python into the NeMo Relay binary.

Managed Surfaces Versus Request Defaults

Both remote mode and local mode share the same top-level plugin model, but they do not implement every part of that model in the same way.

At the plugin-model level, NeMo Guardrails uses two different concepts:

  • Top-level managed NeMo Relay execution surfaces:
    • input
    • output
    • tool_input
    • tool_output
  • Guardrails backend request defaults:
    • request_defaults.context
    • request_defaults.thread_id
    • request_defaults.state
    • request_defaults.rails
    • request_defaults.llm_params
    • request_defaults.llm_output
    • request_defaults.output_vars
    • request_defaults.log

This distinction matters:

  • Managed surfaces wrap real NeMo Relay execution boundaries such as llm.execute(...) and tools.execute(...).
  • Managed surfaces give NeMo Relay an owned enforcement point around a known runtime step. Depending on the backend and surface, Relay can block work, allow it, or apply managed request or result handling before the application sees the outcome.
  • Managed surfaces also give NeMo Relay a stable runtime boundary for its own middleware ordering, lifecycle behavior, and observability marks.

The forwarded request-default side is more mode-specific:

  • In remote mode, request_defaults fields are forwarded to the selected Guardrails backend as request semantics. They do not create new NeMo Relay-native execution surfaces.
  • In local mode, request_defaults is rejected instead of passed through.

The overlap in names is important in remote mode:

  • Top-level input is a managed NeMo Relay execution surface.
  • request_defaults.rails.input is a backend pass-through option.
  • Top-level output is a managed NeMo Relay execution surface.
  • request_defaults.rails.output is a backend pass-through option.
  • Top-level tool_input is a managed NeMo Relay execution surface in the plugin contract. The current stock-remote backend rejects it, while the local backend supports it.
  • request_defaults.rails.tool_input is a backend pass-through option.
  • Top-level tool_output is a managed NeMo Relay execution surface.
  • request_defaults.rails.tool_output is a backend pass-through option.

In particular, request_defaults.rails.dialog and request_defaults.rails.retrieval are pass-through options. They are not separate managed middleware surfaces in NeMo Relay.

Pages