> This page is for version v0.25.0.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo/guardrails/latest/llms.txt
> - v0.25.0: https://docs.nvidia.com/nemo/guardrails/v0.25.0/llms.txt
> - v0.24.1: https://docs.nvidia.com/nemo/guardrails/v0.24.1/llms.txt
> - v0.24.0: https://docs.nvidia.com/nemo/guardrails/v0.24.0/llms.txt
> - v0.23.0: https://docs.nvidia.com/nemo/guardrails/v0.23.0/llms.txt
> - v0.22.0: https://docs.nvidia.com/nemo/guardrails/v0.22.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.

# About Running Guardrailed Inference

> Run guardrailed inference using the Python API or Guardrails API server, apply checks around existing model APIs with NeMo Guardrails Gateway, or check coding agents' tool calls with the hooks server.

After you [configure your guardrails](/nemo/guardrails/v0.25.0/configure-guardrails/configure-rails), you can run guardrailed inference using the tools provided by the NeMo Guardrails library: the Python API and the Guardrails API server.

These tools enable you to interact with your application's main LLM as usual by sending prompts and receiving responses while the guardrails system monitors and controls all communication in the background.
The guardrails intercept inputs and outputs between uses and the LLM and the execution of actions done by the LLM. The tools apply your configured guardrails and ensure that any inputs, generated responses, or actions remain within the boundaries you defined.

To guard a coding agent such as Claude Code or Codex, which runs its own model, use the Guardrails hooks server instead: it checks each tool call the agent is about to run.

To apply checks around supported existing model HTTP exchanges, use
[NeMo Guardrails Gateway](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway).

---

## Install the NeMo Guardrails Library

To use the Python API or the Guardrails API server, you need to install the NeMo Guardrails library. See [Installation](/nemo/guardrails/v0.25.0/get-started/installation-guide) for instructions.

---

## Choosing the Right Tool

Both the Python API and the Guardrails API server are production-ready approaches for integrating guardrails into your application.
The Guardrails hooks server is for coding agents, which you connect through their hook settings rather than your own code.

Choose an interface based on the following tasks:

| Task                                                                                                        | Interface                                                                |
| ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| Call guardrails from Python code in your application                                                        | The Python API                                                           |
| Send chat requests or checks over HTTP, from any language                                                   | The Guardrails API server (`nemoguardrails server`)                      |
| Check supported existing provider HTTP exchanges while preserving accepted JSON and server-sent event bytes | NeMo Guardrails Gateway (`python -m nemoguardrails.server.experimental`) |
| Check a coding agent's tool calls before they run, from the agent's own hooks                               | The Guardrails hooks server (`nemoguardrails hooks-server`)              |

### Python API for Edge and Embedded Applications

Build guardrails directly into your Python application. You call guardrails functions in your code, and everything runs in the same process with no network overhead.

```python
from nemoguardrails import Guardrails, RailsConfig

config = RailsConfig.from_path("path/to/config")
rails = Guardrails(config)

response = rails.generate(messages=[
    {"role": "user", "content": "Hello!"}
])
```

> **Warning**
>
> `LLMRails`, which earlier versions of this example used, is deprecated in 0.25.0; removal is planned no earlier than 0.27.0.
> `Guardrails` runs the IORails engine when that engine supports the configuration.
> For the steps, refer to [Migrating to 0.25](/nemo/guardrails/v0.25.0/reference/0-25#replace-llmrails-with-guardrails).

This approach is best for:

* Edge deployments where no networked backend is available.
* Applications that require low latency and no external dependencies.
* Rapid prototyping and development in notebooks or scripts.
* Fine-grained control over generation and custom streaming handlers.

### Guardrails API Server for Networked and Multi-Client Applications

The Guardrails API server provides a RESTful API interface to the guardrails library. Instead of calling Python functions directly, you make HTTP requests to the server endpoints, making integration straightforward for any language or platform.

```bash
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "meta/llama-3.1-8b-instruct", "messages": [{"role": "user", "content": "Hello!"}], "guardrails": {"config_id": "content_safety"}}'
```

This approach is best for:

* Networked systems where multiple clients need access to guardrails.
* Non-Python clients such as JavaScript, Go, and Java that communicate through HTTP.

### NeMo Guardrails Gateway for Existing Provider Exchanges

Use [NeMo Guardrails Gateway](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway) when your
application already calls a model service over HTTP. The gateway checks inputs
and outputs for supported Chat Completions or Responses traffic using the selected
Rails configuration, and forwards generation to the configured upstream.

The current bindings support one user text input and one assistant text target;
they do not cover arbitrary conversation histories, tools, or multimodal content.
Use its own manifest, library `server` extra, and module launcher, as described
in the [quickstart](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/quickstart).

### Guardrails Hooks Server for Coding Agents

The Guardrails hooks server is a separate server that checks the tool calls of Claude Code, Codex, and NeMo Relay before they run.
The agent's own pre-tool hook sends each call to the server, which applies a tool policy and per-tool rails and answers with allow or deny.

```bash
nemoguardrails hooks-server --config examples/configs/agent_hooks/claude_code
```

This approach is best for:

* Coding agents whose code you do not control, and that can call out only from a hook.
* Blocking risky tools, arguments, or commands, such as a force push or a read of a secrets file, before the agent runs them.

---

## Next Steps

After choosing the right tool for your implementation model, proceed to the corresponding guide from the following.

#### [Python API](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis)

Run guardrailed inference using the NeMo Guardrails Python API.

Get Started

#### [Guardrails API Server](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server)

Expose guardrails through an HTTP API using the Guardrails API server.

Get Started

#### [NeMo Guardrails Gateway](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway)

Apply input and output checks around supported provider JSON and server-sent event exchanges.

Get Started

#### [Guardrails Hooks Server](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server)

Check coding agents' tool calls before they run, using the Guardrails hooks server.

Get Started