ObservabilityMetrics

Metric Reference

View as Markdown

This page lists every metric the IORails engine emits when metrics.enabled: true and a MeterProvider is configured.

Metrics fall into two families:

  • Request-level metrics (guardrails.*) describe IORails request flow: volume, errors, blocks, latency, and saturation of the streaming and non-streaming admission paths.
  • LLM client-side metrics (gen_ai.client.*) describe downstream LLM calls IORails issues. These follow the OpenTelemetry GenAI semantic conventions and use the bucket boundaries recommended by that spec.

Request-Level Metrics

MetricInstrumentUnitLabelsDescription
guardrails.requestsCounter1—Total IORails requests handled, incremented on entry.
guardrails.requests.errorsCounter1error.typeRequests that ended in an unhandled error. error.type is the exception class name (for example QueueFull, TimeoutError).
guardrails.requests.blockedCounter1rail.typeRequests blocked by an input or output rail. rail.type is input or output.
guardrails.request.durationHistograms—End-to-end request duration. For non-streaming requests this includes queue-wait time.
guardrails.requests.activeUpDownCounter1—Requests currently in flight. Covers both streaming and non-streaming.

Bucket Boundaries: guardrails.request.duration

The duration histogram buckets use seconds:

[0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1.0, 2.5, 5.0, 7.5, 10.0]

Saturation Metrics

These metrics expose the internal admission paths so you can detect overload before users encounter errors.

Non-Streaming Path (Admission Queue)

MetricInstrumentUnitDescription
guardrails.nonstream.queuedObservableGauge1Requests buffered in the admission queue, not yet picked up by a worker.
guardrails.nonstream.activeObservableGauge1Requests currently executing on a worker.
guardrails.nonstream.rejectionsCounter1Submissions rejected with QueueFull because the admission queue exceeded its depth limit.

queued and active are read live at collection time, so dashboards always show the current state. After IORails.stop() is called, both gauges return no observations rather than stale values.

Streaming Path (Concurrency Semaphore)

MetricInstrumentUnitDescription
guardrails.stream.activeUpDownCounter1In-progress streaming requests holding a semaphore permit.
guardrails.stream.rejectionsCounter1Streaming requests rejected because the streaming concurrency semaphore was fully occupied.

Cross-Checking Saturation Metrics

At any collection instant, the sum of the per-path saturation gauges should approximately equal guardrails.requests.active:

guardrails.requests.active ≈ guardrails.nonstream.queued
                           + guardrails.nonstream.active
                           + guardrails.stream.active

A persistent drift between the two is a useful integrity check during dashboard development.

Dual-Counted Rejections

A QueueFull rejection on the non-streaming path increments both:

  • guardrails.nonstream.rejections (saturation signal)
  • guardrails.requests.errors{error.type=QueueFull} (error signal)

This is intentional: dashboards built around either signal alone still reflect the rejection.

LLM Client-Side Metrics

These metrics are recorded once per downstream LLM call, not once per IORails request, and follow the OpenTelemetry GenAI semantic conventions.

MetricInstrumentUnitLabelsDescription
gen_ai.client.token.usageHistogram{token}gen_ai.operation.name, gen_ai.provider.name, gen_ai.request.model, gen_ai.token.typeNumber of tokens consumed by an LLM call. Each call records two observations distinguished by the required gen_ai.token.type label (input or output).
gen_ai.client.operation.durationHistogramsgen_ai.operation.name, gen_ai.provider.name, gen_ai.request.model, optionally error.typeWall-clock duration of one LLM call. error.type is added as a conditional label only when the call raises.
gen_ai.client.operation.time_to_first_chunkHistogramsgen_ai.operation.name, gen_ai.provider.name, gen_ai.request.modelStreaming-only. Time from request issue to the first content-bearing chunk.
gen_ai.client.operation.time_per_output_chunkHistogramsgen_ai.operation.name, gen_ai.provider.name, gen_ai.request.modelStreaming-only. Inter-chunk gap; one observation per content-bearing chunk after the first.

gen_ai.token.type only takes the values input and output per spec. Reasoning and cached tokens are exposed as span attributes (gen_ai.usage.reasoning.output_tokens and so on), not as additional metric label values.

Bucket Boundaries

Per the OpenTelemetry GenAI spec, durations use powers-of-two boundaries up to ~82 s:

[0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56, 5.12, 10.24, 20.48, 40.96, 81.92]

Token counts use powers-of-four boundaries up to ~67M tokens:

[1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864]

Both match the spec exactly so backends auto-render the distributions correctly.

Streaming vs. Non-Streaming Emission

MetricNon-streamingStreaming
gen_ai.client.token.usage✓✓ (when the upstream provider returns usage)
gen_ai.client.operation.duration✓✓
gen_ai.client.operation.time_to_first_chunk—✓
gen_ai.client.operation.time_per_output_chunk—✓

For streaming responses, token.usage is emitted only when the upstream provider returns a usage field. This is common when stream_options.include_usage=true is forwarded. When usage is absent, no observation is recorded; “no observation” is deliberately distinct from “0 tokens”.

Common Label Reference

LabelUsed OnValuesNotes
error.typeguardrails.requests.errors, gen_ai.client.operation.duration (on error)Exception class nameFor example QueueFull, TimeoutError, ValueError.
rail.typeguardrails.requests.blockedinput, outputIdentifies whether an input or output rail blocked the request.
gen_ai.operation.nameAll gen_ai.client.*For example chat, completion, embeddingOpenTelemetry GenAI operation name.
gen_ai.provider.nameAll gen_ai.client.*For example openai, anthropicOpenTelemetry GenAI provider name.
gen_ai.request.modelAll gen_ai.client.*For example gpt-4o-miniThe model name passed in the request.
gen_ai.token.typegen_ai.client.token.usageinput, outputRequired label per spec.

Public API Stability

The metric names listed on this page are part of the library’s public API, so dashboards and alerts can reference them. The library tests assert on the raw strings for this reason. Bucket boundaries follow the OpenTelemetry GenAI spec and can change if the spec changes.