OpenTelemetry Metrics Integration

The NVIDIA NeMo Guardrails library follows OpenTelemetry best practices. The library uses only the API, and the host application configures the SDK. The following sections explain how to install and configure the OpenTelemetry SDK for metrics export from the NeMo Guardrails IORails engine.

Installation

Choose one of the following options for installing the library, the OpenTelemetry SDK, and an exporter.

For development with the OpenTelemetry SDK (console exporter only):

$ pip install "nemoguardrails[tracing]" opentelemetry-sdk

For production with the OpenTelemetry Protocol (OTLP) exporter:

$ pip install "nemoguardrails[tracing]" opentelemetry-sdk opentelemetry-exporter-otlp

For Prometheus scraping:

$ pip install "nemoguardrails[tracing]" opentelemetry-sdk opentelemetry-exporter-prometheus prometheus-client

prometheus-client is required because the example uses start_http_server to expose the scrape endpoint.

Enabling Metrics in the Configuration

Set metrics.enabled: true in your guardrails configuration. This is independent of tracing.enabled.

1 metrics:
2   enabled: true

When metrics.enabled is true and opentelemetry-api is installed, the IORails engine emits metrics through the active MeterProvider. When opentelemetry-api is not installed, the IORails engine emits a UserWarning at construction time and runs without metrics.

Configuration Examples

Console Output (Development)

Use the console exporter to verify emissions locally before pointing at a real backend.

1 from opentelemetry import metrics
2 from opentelemetry.sdk.metrics import MeterProvider
3 from opentelemetry.sdk.metrics.export import (
4     ConsoleMetricExporter,
5     PeriodicExportingMetricReader,
6 )
7 from opentelemetry.sdk.resources import Resource
8 
9 resource = Resource.create({"service.name": "my-guardrails-app"})
10 reader = PeriodicExportingMetricReader(
11     ConsoleMetricExporter(),
12     export_interval_millis=5000,
13 )
14 metrics.set_meter_provider(MeterProvider(resource=resource, metric_readers=[reader]))
15 
16 # Configure NeMo Guardrails afterwards.
17 from nemoguardrails import Guardrails, RailsConfig
18 
19 config_yaml = """
20 models:
21   - type: main
22     engine: openai
23     model: gpt-4o-mini
24 
25 metrics:
26   enabled: true
27 """
28 
29 config = RailsConfig.from_content(yaml_content=config_yaml)
30 # require_iorails=True raises if the config is incompatible with the IORails engine,
31 # so the metrics setup fails loudly rather than silently falling back to LLMRails
32 # (which emits no metrics).
33 rails = Guardrails(config, use_iorails=True, require_iorails=True)

OTLP Exporter (Production)

Push metrics over OTLP/gRPC to an OpenTelemetry Collector or any backend that accepts OTLP.

1 from opentelemetry import metrics
2 from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
3 from opentelemetry.sdk.metrics import MeterProvider
4 from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
5 from opentelemetry.sdk.resources import Resource
6 
7 resource = Resource.create({"service.name": "my-guardrails-app"})
8 
9 otlp_exporter = OTLPMetricExporter(endpoint="http://localhost:4317", insecure=True)
10 reader = PeriodicExportingMetricReader(
11     otlp_exporter,
12     export_interval_millis=10000,
13 )
14 metrics.set_meter_provider(MeterProvider(resource=resource, metric_readers=[reader]))
15 
16 # Use with NeMo Guardrails as above.

Prometheus Exporter

Use the Prometheus exporter to expose metrics on an HTTP endpoint that Prometheus scrapes directly.

1 from prometheus_client import start_http_server
2 
3 from opentelemetry import metrics
4 from opentelemetry.exporter.prometheus import PrometheusMetricReader
5 from opentelemetry.sdk.metrics import MeterProvider
6 from opentelemetry.sdk.resources import Resource
7 
8 start_http_server(port=9464, addr="0.0.0.0")
9 
10 resource = Resource.create({"service.name": "my-guardrails-app"})
11 reader = PrometheusMetricReader()
12 metrics.set_meter_provider(MeterProvider(resource=resource, metric_readers=[reader]))
13 
14 # Use with NeMo Guardrails as above.

After Prometheus has scraped the endpoint, periods in OpenTelemetry metric names are replaced with underscores, for example guardrails.requests becomes guardrails_requests_total and gen_ai.client.token.usage becomes gen_ai_client_token_usage_bucket. Refer to the Prometheus exposition format for details on how dots, units, and counter suffixes are translated.

OpenTelemetry Ecosystem Compatibility

You can export IORails metrics to any backend that accepts OpenTelemetry metric data, including:

OTLP-compatible backends, such as New Relic, Datadog, Honeycomb, Grafana Cloud, AWS CloudWatch (with the ADOT collector), and Google Cloud Monitoring.
Prometheus and any system that ingests Prometheus exposition format (Grafana Mimir, Cortex, Thanos, VictoriaMetrics).
OpenTelemetry Collector as a fan-out point to receive metrics once and export them to many backends.

Refer to the OpenTelemetry Registry for the complete list.

Combining Metrics with Tracing

Configure each signal independently. Metrics and traces correlate naturally when both are exported under the same service.name resource: trace exemplars on histograms link directly to their request spans.

1 tracing:
2   enabled: true
3   adapters:
4     - name: OpenTelemetry
5 
6 metrics:
7   enabled: true

1 from opentelemetry import metrics, trace
2 from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
3 from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
4 from opentelemetry.sdk.metrics import MeterProvider
5 from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
6 from opentelemetry.sdk.resources import Resource
7 from opentelemetry.sdk.trace import TracerProvider
8 from opentelemetry.sdk.trace.export import BatchSpanProcessor
9 
10 # Application-side: configure a TracerProvider AND a MeterProvider with the same Resource.
11 resource = Resource.create({"service.name": "my-guardrails-app"})
12 
13 # 1. Tracing
14 tracer_provider = TracerProvider(resource=resource)
15 tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4317", insecure=True)))
16 trace.set_tracer_provider(tracer_provider)
17 
18 # 2. Metrics
19 reader = PeriodicExportingMetricReader(OTLPMetricExporter(endpoint="http://localhost:4317", insecure=True))
20 metrics.set_meter_provider(MeterProvider(resource=resource, metric_readers=[reader]))

For full tracing setup, refer to OpenTelemetry.