Enable Guardrails Metrics

Use this minimal setup to enable metrics from IORails with the OpenTelemetry SDK and console output. LLMRails does not support OpenTelemetry metrics. Use this to verify metric emission locally before wiring up a production exporter.

1. Install the NeMo Guardrails library and the OpenTelemetry SDK.

   $
   pip install "nemoguardrails[tracing]" opentelemetry-sdk

   The [tracing] extra installs opentelemetry-api, which is the only OpenTelemetry dependency the library itself takes.

2. Save the following to metrics_example.py.

   The script issues a single request and exports metrics once per second to both stdout and metrics.json. The provider.shutdown() call flushes the metrics to disk. Long-running services typically do not need this call.

   1
   # metrics_example.py
   2
   import asyncio
   3
   4
   from opentelemetry import metrics
   5
   from opentelemetry.sdk.metrics import MeterProvider
   6
   from opentelemetry.sdk.metrics.export import (
   7
       ConsoleMetricExporter,
   8
       PeriodicExportingMetricReader,
   9
   )
   10
   from opentelemetry.sdk.resources import Resource
   11
   12
   from nemoguardrails import Guardrails, RailsConfig
   13
   # Configure the OpenTelemetry MeterProvider BEFORE constructing Guardrails so
   14
   # the engine resolves a real meter on first metric emission.  Two readers
   15
   # are attached to the same provider: one writes to stdout for live
   16
   # inspection, the other writes to ``metrics.json``.
   17
   resource = Resource.create({"service.name": "guardrails-quickstart"})
   18
   metrics_file = open("metrics.json", "w")
   19
   console_reader = PeriodicExportingMetricReader(
   20
       ConsoleMetricExporter(),
   21
       export_interval_millis=1000,
   22
   )
   23
   file_reader = PeriodicExportingMetricReader(
   24
       ConsoleMetricExporter(out=metrics_file),
   25
       export_interval_millis=1000,
   26
   )
   27
   provider = MeterProvider(
   28
       resource=resource,
   29
       metric_readers=[console_reader, file_reader],
   30
   )
   31
   metrics.set_meter_provider(provider)
   32
   33
   # Use Configuration with metrics enabled.
   34
   config_yaml = """
   35
   models:
   36
     - type: main
   37
       engine: openai
   38
       model: gpt-4o-mini
   39
   40
   metrics:
   41
     enabled: true
   42
   """
   43
   44
   config = RailsConfig.from_content(yaml_content=config_yaml)
   45
   46
   async def main() -> None:
   47
       # use_iorails=True selects the IORails engine (the only engine that emits
   48
       # metrics). require_iorails=True raises a ValueError if the config is
   49
       # incompatible with IORails.
   50
       async with Guardrails(config, use_iorails=True, require_iorails=True) as rails:
   51
           response = await rails.generate_async(
   52
               messages=[{"role": "user", "content": "Write an essay with historical context on NVIDIA"}],
   53
           )
   54
           print(f"Response: {response}")
   55
   56
   try:
   57
       asyncio.run(main())
   58
   finally:
   59
       # Flush and tear down the reader so the final batch reaches the exporter
   60
       # before the process exits, then close the metrics file.
   61
       provider.shutdown()
   62
       metrics_file.close()

3. Run the script.

   $
   python metrics_example.py

4. Post-process the metrics JSON file.

   metrics.json contains one JSON document per export interval, concatenated. The following command reads every interval into an array and returns the final object, which is the cumulative state after the request completed.

   $
   jq -s 'last | .resource_metrics[].scope_metrics[].metrics[]
   >
          | select(.name=="gen_ai.client.token.usage")
   >
          | .data.data_points[]
   >
          | {type: .attributes."gen_ai.token.type", count, sum}' metrics.json

   Example output (exact output token counts can vary):

   1
   {
   2
     "type": "input",
   3
     "count": 1,
   4
     "sum": 15
   5
   }
   6
   {
   7
     "type": "output",
   8
     "count": 1,
   9
     "sum": 1033
   10
   }

   Each object is one aggregation of the gen_ai.client.token.usage histogram. The fields are:

   • type: Value of the required gen_ai.token.type label. Values are input or output.
   • count: Number of observations recorded for this token type. Each LLM call records one input observation and one output observation, giving count: 1 per type.
   • sum: Total tokens across those observations.

Next Steps

• For production exporters (OTLP, Prometheus), refer to OpenTelemetry Metrics Integration.
• For the full list of emitted metrics, refer to Metric Reference.