> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/guardrails/_mcp/server.

# Architecture Overview

> Understand the request flow, runtime layers, and integration points in the NVIDIA NeMo Guardrails library.

The NVIDIA NeMo Guardrails library sits between your application and the LLM, tool, or retrieval systems that the application uses. It loads a guardrails configuration, evaluates incoming user messages against the configured rails, orchestrates any required LLM calls or custom actions, and returns either an approved response or a configured safe alternative.

Use this page to understand the guardrails process at a high level. For deeper event-level details, refer to [Colang Architecture Guide](../reference/colang-architecture-guide), [Guardrails Sequence Diagrams](../reference/guardrails-sequence-diagrams), and [Use Case Diagrams](../reference/use-case-diagrams).

## High-Level Architecture Diagram

The following diagram shows how guardrails intervene in the application's request flow.

![Programmable Guardrails Flow](file:_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/private/var/folders/gv/z1x0xgb54bj3k7vlncdh6pnc0000gp/T/tmp-13788-myEg1gmTpeXQ/docs/_static/images/programmable_guardrails.png "Programmable Guardrails Flow")

## Functional Layers

The NVIDIA NeMo Guardrails library separates application integration, policy configuration, runtime orchestration, and external systems. This separation lets your application call one guardrailed interface while the library handles the configured checks and workflows.

| Layer                        | Role                                                                                                         |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------ |
| **Application integration**  | Sends messages through the Python SDK or the guardrails server and receives the final response.              |
| **Guardrails configuration** | Defines models, prompts, rails, Colang flows, knowledge base settings, and custom actions.                   |
| **Runtime orchestration**    | Loads the configuration, processes events, executes rails, calls actions, and coordinates LLM requests.      |
| **External systems**         | Provide LLM inference, retrieval content, moderation services, tool calls, or other custom validation logic. |

## Request Flow

When an application sends a user message, the NVIDIA NeMo Guardrails library evaluates the message in the configured order:

1. **Input rails** inspect the user message before the application LLM is called. These rails can allow, alter, or reject the message.
2. **Retrieval rails** run when the configuration uses a knowledge base or retrieved context. These rails can filter or transform retrieved chunks before they enter the prompt.
3. **Dialog rails** use Colang flows and runtime events to guide the conversation, choose the next step, or decide whether the bot should respond or execute an action.
4. **Execution rails** validate and control custom actions or tools, including their inputs and outputs.
5. **Output rails** inspect the generated response before it returns to the user. These rails can allow, edit, or block the response.

For more information about where each rail type runs, refer to [Guardrail Types](/about-nemo-guardrails-library/rail-types).

## Runtime Behavior

The runtime uses an event-driven design. A user message becomes an event, and the runtime processes that event through the active Colang flows and configured rails. Depending on the configuration, the runtime can generate a canonical user intent, decide the next step, execute a custom action, retrieve knowledge base chunks, call the application LLM, or generate the final bot message.

Applications do not need to implement this orchestration themselves. They call the NVIDIA NeMo Guardrails library through the SDK or server, and the library coordinates the underlying workflow.

## External Integration Points

The NVIDIA NeMo Guardrails library can work with several external systems.

| Integration                       | Details                                                                                                                        |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| **Application LLM**               | The model that generates responses for your application.                                                                       |
| **Guardrail models and services** | Specialized models, NVIDIA NIM microservices, or third-party APIs that perform safety, jailbreak, topic, PII, or other checks. |
| **Knowledge bases**               | Documents or chunks that support retrieval-augmented generation workflows.                                                     |
| **Custom actions and tools**      | Python functions, LangChain tools, or external APIs that add application-specific logic.                                       |

## Related Resources

Use the following resources to learn more about the NVIDIA NeMo Guardrails library:

* [Guardrail Types](/about-nemo-guardrails-library/rail-types)
* [Get Started](/get-started/installation-guide)
* [Guardrail Catalog](/configure-guardrails/guardrail-catalog)