About NeMo Guardrails Library

Architecture Overview

View as Markdown

The NVIDIA NeMo Guardrails library sits between your application and the LLM, tool, or retrieval systems that the application uses. It loads a guardrails configuration, evaluates incoming user messages against the configured rails, orchestrates any required LLM calls or custom actions, and returns either an approved response or a configured safe alternative.

Use this page to understand the guardrails process at a high level. For deeper event-level details, refer to Colang Architecture Guide, Guardrails Sequence Diagrams, and Use Case Diagrams.

High-Level Architecture Diagram

The following diagram shows how guardrails intervene in the application’s request flow.

Programmable Guardrails Flow

Functional Layers

The NVIDIA NeMo Guardrails library separates application integration, policy configuration, runtime orchestration, and external systems. This separation lets your application call one guardrailed interface while the library handles the configured checks and workflows.

LayerRole
Application integrationSends messages through the Python SDK or the guardrails server and receives the final response.
Guardrails configurationDefines models, prompts, rails, Colang flows, knowledge base settings, and custom actions.
Runtime orchestrationLoads the configuration, processes events, executes rails, calls actions, and coordinates LLM requests.
External systemsProvide LLM inference, retrieval content, moderation services, tool calls, or other custom validation logic.

Request Flow

When an application sends a user message, the NVIDIA NeMo Guardrails library evaluates the message in the configured order:

  1. Input rails inspect the user message before the application LLM is called. These rails can allow, alter, or reject the message.
  2. Retrieval rails run when the configuration uses a knowledge base or retrieved context. These rails can filter or transform retrieved chunks before they enter the prompt.
  3. Dialog rails use Colang flows and runtime events to guide the conversation, choose the next step, or decide whether the bot should respond or execute an action.
  4. Execution rails validate and control custom actions or tools, including their inputs and outputs.
  5. Output rails inspect the generated response before it returns to the user. These rails can allow, edit, or block the response.

For more information about where each rail type runs, refer to Guardrail Types.

Runtime Behavior

The runtime uses an event-driven design. A user message becomes an event, and the runtime processes that event through the active Colang flows and configured rails. Depending on the configuration, the runtime can generate a canonical user intent, decide the next step, execute a custom action, retrieve knowledge base chunks, call the application LLM, or generate the final bot message.

Applications do not need to implement this orchestration themselves. They call the NVIDIA NeMo Guardrails library through the SDK or server, and the library coordinates the underlying workflow.

External Integration Points

The NVIDIA NeMo Guardrails library can work with several external systems.

IntegrationDetails
Application LLMThe model that generates responses for your application.
Guardrail models and servicesSpecialized models, NVIDIA NIM microservices, or third-party APIs that perform safety, jailbreak, topic, PII, or other checks.
Knowledge basesDocuments or chunks that support retrieval-augmented generation workflows.
Custom actions and toolsPython functions, LangChain tools, or external APIs that add application-specific logic.

Use the following resources to learn more about the NVIDIA NeMo Guardrails library: