About NeMo Guardrails Library

Release Notes

View as Markdown

The following sections summarize and highlight the changes for each release. For a complete record of changes in a release, refer to the CHANGELOG.md in the GitHub repository.

0.23.0

Key Features

  • IORails adds tool-calling rails for OpenAI Chat Completions-style tool traffic. The engine can validate model-emitted tool calls and application tool results in streaming and non-streaming flows, without making extra LLM or provider calls. For more information, refer to Tool Calling and Engine Feature Support.

  • The guardrail catalog adds Polygraf PII detection and masking integration. You can configure Polygraf for input, output, and retrieval flows, including fail-closed handling for provider failures or malformed entity spans. For more information, refer to Polygraf Integration.

  • The library adds lightweight Hugging Face classifier rails and a context bloat detection rail. The classifier rails can run on input, output, or retrieval text with local Transformers, vLLM, KServe, or FMS backends. The context bloat rail detects padded, oversized, low-entropy, or repetitive user and retrieval text before it can consume context budget.

  • The Guardrails server adds /v1/checks for standalone guardrail validation. The endpoint accepts stored or inline configurations, runs check_async, and returns whether the messages passed, were modified, or were blocked, along with the processed content and blocking rail when available.

  • Observability adds opt-in OpenTelemetry content capture and richer LLM span attributes. You can record request, response, rail, and model-call content when tracing is enabled, and LLM spans now include request, response, and usage attributes for both LLMRails and IORails. For more information, refer to Capturing Prompt and Response Content and Span Reference.

Breaking Changes

  • The NVIDIA NeMo Guardrails library now requires Pydantic >=2.5,<3.0. Configuration schemas and validators use Pydantic 2 APIs. Update custom code that imports, subclasses, or depends on Pydantic 1 behavior from the library’s configuration models before upgrading.

Enhancements

  • The default embedding search implementation now uses exact NumPy search instead of Annoy. Existing default knowledge-base caches are regenerated automatically, and the change removes Annoy’s native extension from the default dependency path. For more information, refer to Embedding Search Providers.

  • OpenAI Responses API guidance and compatibility are improved for the LangChain framework path. Function tool calls are surfaced in streaming and non-streaming modes, and Harmony response format models can use the Responses API configuration. For more information, refer to Model Configuration.

  • The public API is refined around the modern Guardrails facade and documented top-level imports. Direct access to internal LLMRails attributes is deprecated so applications can move toward stable public entry points.

  • The Python package is smaller because examples, repository files, and agent instruction files are no longer bundled in the wheel or source distribution. The release also introduces NIM-based example notebooks and retires superseded examples.

Documentation and Behavior Fixes

  • Streaming output rails now pass user content correctly, avoid duplicate usage metadata chunks, and do not reuse resolved action parameters across chunks or requests. Regex detection during output streaming now blocks matched content correctly without raising TypeError. For more information, refer to Streaming.

  • Tool-call metadata is preserved for LLMRails tool rails and processing logs, so tool-aware integrations can inspect tool calls after guardrail execution.

  • Configuration and cache handling are more deterministic. Library files load in a stable order, EmbeddingsCache.from_dict preserves store_config, and evaluation YAML handling uses safe load and dump behavior.

  • Colang and generation fixes cover multiline bot say flow continuations, or line continuations at the end of a file, and task-specific stop-token selection in generate_value.

0.22.0

Key Features

  • LangChain is now optional. pip install nemoguardrails no longer pulls LangChain or any provider-specific langchain-* packages. The NVIDIA NeMo Guardrails library ships with a built-in client that talks to OpenAI-compatible endpoints directly over httpx. Engines whose API isn’t OpenAI-compatible (Anthropic, Cohere, Vertex AI, Google Generative AI, in-process Hugging Face, TensorRT-LLM, and others) keep working through LangChain when you opt in with NEMOGUARDRAILS_LLM_FRAMEWORK=langchain and install the matching provider package. Most 0.21 configurations keep working unchanged; some shapes need a YAML rewrite. For recipes, refer to Migrating to v0.22.0, the Supported LLMs matrix, and Model Configuration.

  • OpenAI-compatible service support is improved in the default framework. The default framework now supports OpenAI-compatible providers directly, includes native Azure OpenAI support through engine: azure and engine: azure_openai, and documents how to migrate provider-specific LangChain parameters to the new base_url-based configuration shape. For more information, refer to Migrating to v0.22.0, Model Configuration, Configuration Reference, and Using Docker.

  • IORails adds streaming support, reasoning-model support, and speculative generation support. The optimized input and output rails engine now supports streaming output rails, stream_async() integration in chat and server flows, non-streaming and streaming reasoning-model responses, and speculative generation for non-streaming generate_async() calls. For more information, refer to Parallel Rails, Streaming, and Speculative Generation.

  • IORails adds OpenTelemetry observability with logging, tracing, and metrics support. The documentation covers OTLP setup, Prometheus client installation, request-level and token-level metrics, and the recommended Guardrails entry point for the optimized input and output rails engine. For more information, refer to Observability, OpenTelemetry Logs, OpenTelemetry Tracing, OpenTelemetry Metrics, Enable Metrics, and the Metrics Reference.

  • Anonymous usage reporting is documented with clear privacy boundaries and opt-out controls. The telemetry reference explains what fields are collected, what data is excluded, how local audit files work, and how to opt out with NEMO_GUARDRAILS_NO_USAGE_STATS=1, DO_NOT_TRACK=1, or the ~/.config/nemoguardrails/do_not_track file. For more information, refer to Telemetry.

Breaking Changes

  • Moved AsyncWorkQueue from the top-level Guardrails object to IORails. This removes buffering for non-streaming LLMRails requests when you use the top-level Guardrails object. This change only affects existing implementations that set NEMO_GUARDRAILS_IORAILS_ENGINE=1 or instantiate Guardrails directly.

Enhancements

  • The GLiNER PII connector documentation and notebook are updated for the new GLiNER PII NIM. The examples cover both remote and local deployment modes and API key configuration for the connector. For more information, refer to GLiNER and PII Detection.

  • Public extension points for LLM integration. Two new protocols, LLMModel and LLMFramework in nemoguardrails.types, let you plug in a custom backend or a whole alternative framework without touching internals. For more information, refer to Custom LLM Models and Custom LLM Frameworks.

  • Public testing surface. The nemoguardrails.testing module exposes FakeLLMModel, TestChat, and pytest fixtures for writing tests against a guardrails configuration without calling a real model.

Documentation and Behavior Fixes

  • Fixed the example query and expected output in the Guardrails Agent Middleware integration guide so the example matches the configured blocked response behavior. For more information, refer to Guardrails Agent Middleware.
  • A warning about a missing main LLM is now emitted only when generation is actually attempted and the generation path needs the main LLM. Check-only configurations no longer emit the warning during initialization. For more information, refer to Check Messages.
  • Fixed issues in the Colang 1.0 Hello World tutorial and companion notebook.

Previous Releases