> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/llms.txt. # Run Guardrailed Inference ## Docs - [About Running Guardrailed Inference](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/run-rails.md): Run guardrailed inference using the Python API or Guardrails API server, apply checks around existing model APIs with NeMo Guardrails Gateway, or check coding agents' tool calls with the hooks server. - [Use the NeMo Guardrails Python APIs](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis.md): Run guardrailed inference using the NeMo Guardrails Python API. - [Overview of the NeMo Guardrails Python APIs](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis/overview.md): RailsConfig and Guardrails core classes for generating guarded responses. - [Core Classes Reference](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis/core-classes.md): RailsConfig and LLMRails class reference for loading and running guardrails. - [Generation Options Reference](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis/generation-options.md): Configure logging, LLM parameters, and rail selection for generation. - [Streaming Responses](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis/streaming.md): Stream LLM responses in real-time with the stream_async method. - [Checking Messages Against Rails](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis/check-messages.md): Validate messages against input, output, and tool rails using check_async and check methods on the LLMRails and IORails engines. - [Event-Based API](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis/event-based-api.md): Use generate_events for low-level control over guardrails execution. - [Use the Guardrails API Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server.md): Expose guardrails through an HTTP API using the Guardrails API server. - [Overview of the NVIDIA NeMo Guardrails Library API Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server/overview.md): Expose configured guardrails and models through the NVIDIA NeMo Guardrails library HTTP API. - [Run the Guardrails Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server/run-guardrails-server.md): Start the Guardrails API server, configure CORS, and enable auto-reload. - [Chat with Guardrailed Model](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server/chat-with-guardrailed-model.md): Send chat requests, use streaming, and manage conversation threads. - [List Guardrail Configurations](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server/list-guardrail-configs.md): Retrieve available guardrails configurations from the server. - [List Available Models](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server/list-models.md): Query the available LLM models from the configured provider. - [Use the Actions Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server/actions-server.md): Run guardrail actions in a separate server process. - [NeMo Guardrails Gateway](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway.md): Apply NeMo Guardrails to existing model APIs with configurable checks that preserve supported provider formats and work with your inference infrastructure. - [Run NeMo Guardrails Gateway](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/quickstart.md): Start NeMo Guardrails Gateway for OpenAI, send buffered and streaming requests from a shell or Python, and block input before inference. - [Configure the Gateway](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/configuration.md): Choose provider instances, operations, dialects, and Rails configurations without coupling routing to inspection policy. - [Providers, Profiles, and Streaming](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/providers-and-streaming.md): Check supported request shapes, select a provider dialect, and tune streamed output checks. - [Understand Guard Contracts](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/guard-contracts.md): Review what a provider binding inspects, constrains, and passes through using contracts exported from its Python declarations. - [Compose and Extend in Python](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/python-integration.md): Build the gateway's HTTP proxy application, select Rails from trusted context, and extend HTTP dispatch or provider bindings. - [Deploy and Route Requests](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/deployment.md): Choose a topology for one upstream, multiple provider instances, replicas, gateways, or embedded applications. - [Troubleshoot NeMo Guardrails Gateway](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/troubleshooting.md): Diagnose startup failures, rejected payloads, checker errors, delayed streams, disconnects, and incomplete cleanup. - [Rails Configuration Resolution](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/advanced/configuration-resolution.md): Select preloaded Rails configurations from trusted request context and manage resolver deadlines and lifecycle. - [HTTP Transport Customization](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/advanced/http-transport.md): Inject outbound HTTP behavior, route model aliases, tune shared pools, and preserve streaming ownership. - [Multiple Provider Instances and Routing](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/advanced/multiple-provider-instances.md): Compose mounts, policies and upstreams while understanding shared resources, route ownership and isolation. - [Customize Request and Response Policies](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/advanced/extend-provider-bindings.md): Customize accepted request values and response metadata for buffered and streaming traffic, then export the bound guard contract. - [Architecture](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/architecture.md): Understand NeMo Guardrails Gateway's boundaries, inspection model, design tradeoffs, and deployment responsibilities. - [Request Lifecycle](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/architecture/request-lifecycle.md): Follow buffered and streamed requests through validation, checking, byte release, errors, and cleanup. - [Framework Internals](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/architecture/framework-internals.md): Understand provider-neutral execution, composition hooks, trusted policy selection, and ownership when extending the gateway. - [Use the Guardrails Hooks Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server.md): Check the tool calls of Claude Code, Codex, and NeMo Relay before they run, with the Guardrails hooks server. - [Overview of the Guardrails Hooks Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/overview.md): The Guardrails hooks server checks a coding agent's tool calls before they run, from the agent's own pre-tool hook. - [Run the Hooks Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/run-hooks-server.md): Start the Guardrails hooks server, write its tool policy and per-tool rails, and monitor it with logs, metrics, and traces. - [Check Claude Code Tool Calls with the Hooks Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/claude-code.md): Connect Claude Code to the Guardrails hooks server with an HTTP PreToolUse hook, so every tool call is checked before it runs. - [Check Codex Tool Calls with the Hooks Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/codex.md): Connect Codex to the Guardrails hooks server with a PreToolUse command hook, so every tool call is checked before it runs. - [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin.md): Build, install, and configure the NVIDIA NeMo Guardrails Plugin for NeMo Relay to check tool calls before they run. - [Check NeMo Relay Tool Calls with the Hooks Server](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/nemo-relay.md): The /v1/hooks/nemo-relay endpoint of the Guardrails hooks server, for a NeMo Relay plugin's tool conditional-execution guardrail. - [Hooks Server Results and Errors](https://docs.nvidia.com/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/results-and-errors.md): The allow, deny, and error outcomes of the Guardrails hooks server, its deny reasons and error responses, and how to make each agent block a call when a check fails.