Observability

Tracing Guardrails

View as Markdown

# Tracing Guardrails

Tracing enhances the observability of guardrails execution. This section explains how to configure tracing with the NVIDIA NeMo Guardrails library.

With tracing, you can:

  • Track which rails are activated during conversations.
  • Monitor LLM calls and their performance.
  • Debug flow execution and identify bottlenecks.
  • Analyze conversation patterns and errors.

Span Formats

Starting with v0.16.0, the NeMo Guardrails library uses OpenTelemetry semantic conventions for Generative AI (GenAI) instead of the legacy span format. This change standardizes observability and improves monitoring for AI workloads.

Reference Documentation:

Supported Formats

The NeMo Guardrails library supports the following formats.

  • OpenTelemetry (opentelemetry) - Recommended default format following OpenTelemetry semantic conventions for GenAI
  • Legacy (legacy) - Previous format with simple metrics dictionary (deprecated)

Configuration

You can control the span format through the tracing configuration:

1tracing:
2    enabled: true
3    span_format: "opentelemetry" # default
4    enable_content_capture: false # default, see privacy considerations below

Key Differences

The following are the key differences between the supported span formats.

OpenTelemetry Format:

  • Follows the standardized semantic conventions
  • Rich-structured attributes and events (e.g., gen_ai.request.model, gen_ai.usage.input_tokens)
  • Enhanced LLM call tracking with provider and model information
  • Support for span events and error tracking
  • Compatible with OpenTelemetry ecosystem tools

Legacy Format:

  • Simple metrics dictionary
  • Minimal overhead with limited observability
  • Deprecated - maintained for backward compatibility only

Important Considerations

Development Status: The OpenTelemetry semantic conventions for GenAI are currently in development and may undergo changes. Consider the following risks:

  • Evolving Standards: Conventions may change as they mature, potentially affecting existing implementations
  • Data Privacy: The enable_content_capture option captures user inputs and model outputs, which can include sensitive information (PII). Only enable it when necessary, and ensure compliance with data protection regulations. See Content Capture for the full contract and the GenAI Events documentation for details
  • Performance Impact: Extensive telemetry collection may impact system performance, especially with large inputs/outputs

Migration Path

Existing configurations will continue to work. However, it is strongly recommended to migrate to the OpenTelemetry format. Migration steps are:

  1. Update your configuration to use span_format: "opentelemetry"
  2. Review your telemetry backends for compatibility with OpenTelemetry conventions
  3. Test thoroughly, as span structures and attribute names have changed
  4. Consider privacy implications if enabling content capture

Contents

  • Quick Start - Minimal setup to enable tracing using the OpenTelemetry SDK
  • Adapters - Detailed configuration for FileSystem, OpenTelemetry, and Custom adapters
  • OpenTelemetry - Production-ready OpenTelemetry setup and ecosystem compatibility
  • OpenTelemetry Logs - Forward guardrails Python logs to OpenTelemetry with trace correlation
  • Span Reference - Every span and attribute the LLMRails and IORails engines emit, including the token-level GenAI attributes on the LLM span
  • Content Capture - Record prompt and response content on spans, with the environment variables that control capture and format
  • Troubleshooting - Common issues and solutions

Jupyter Notebooks