About Running Guardrailed Inference
After you configure your guardrails, you can run guardrailed inference using the tools provided by the NeMo Guardrails library: the Python API and the Guardrails API server.
These tools enable you to interact with your application’s main LLM as usual by sending prompts and receiving responses while the guardrails system monitors and controls all communication in the background. The guardrails intercept inputs and outputs between uses and the LLM and the execution of actions done by the LLM. The tools apply your configured guardrails and ensure that any inputs, generated responses, or actions remain within the boundaries you defined.
To guard a coding agent such as Claude Code or Codex, which runs its own model, use the Guardrails hooks server instead: it checks each tool call the agent is about to run.
To apply checks around supported existing model HTTP exchanges, use NeMo Guardrails Gateway.
Install the NeMo Guardrails Library
To use the Python API or the Guardrails API server, you need to install the NeMo Guardrails library. See Installation for instructions.
Choosing the Right Tool
Both the Python API and the Guardrails API server are production-ready approaches for integrating guardrails into your application. The Guardrails hooks server is for coding agents, which you connect through their hook settings rather than your own code.
Choose an interface based on the following tasks:
Python API for Edge and Embedded Applications
Build guardrails directly into your Python application. You call guardrails functions in your code, and everything runs in the same process with no network overhead.
LLMRails, which earlier versions of this example used, is deprecated in 0.25.0; removal is planned no earlier than 0.27.0.
Guardrails runs the IORails engine when that engine supports the configuration.
For the steps, refer to Migrating to 0.25.
This approach is best for:
- Edge deployments where no networked backend is available.
- Applications that require low latency and no external dependencies.
- Rapid prototyping and development in notebooks or scripts.
- Fine-grained control over generation and custom streaming handlers.
Guardrails API Server for Networked and Multi-Client Applications
The Guardrails API server provides a RESTful API interface to the guardrails library. Instead of calling Python functions directly, you make HTTP requests to the server endpoints, making integration straightforward for any language or platform.
This approach is best for:
- Networked systems where multiple clients need access to guardrails.
- Non-Python clients such as JavaScript, Go, and Java that communicate through HTTP.
NeMo Guardrails Gateway for Existing Provider Exchanges
Use NeMo Guardrails Gateway when your application already calls a model service over HTTP. The gateway checks inputs and outputs for supported Chat Completions or Responses traffic using the selected Rails configuration, and forwards generation to the configured upstream.
The current bindings support one user text input and one assistant text target;
they do not cover arbitrary conversation histories, tools, or multimodal content.
Use its own manifest, library server extra, and module launcher, as described
in the quickstart.
Guardrails Hooks Server for Coding Agents
The Guardrails hooks server is a separate server that checks the tool calls of Claude Code, Codex, and NeMo Relay before they run. The agent’s own pre-tool hook sends each call to the server, which applies a tool policy and per-tool rails and answers with allow or deny.
This approach is best for:
- Coding agents whose code you do not control, and that can call out only from a hook.
- Blocking risky tools, arguments, or commands, such as a force push or a read of a secrets file, before the agent runs them.
Next Steps
After choosing the right tool for your implementation model, proceed to the corresponding guide from the following.