> This page is for version v0.24.1. > For other versions, use one of these documentation indexes: > - Latest (default): https://docs.nvidia.com/nemo/guardrails/latest/llms.txt > - v0.24.1: https://docs.nvidia.com/nemo/guardrails/v0.24.1/llms.txt > - v0.24.0: https://docs.nvidia.com/nemo/guardrails/v0.24.0/llms.txt > - v0.23.0: https://docs.nvidia.com/nemo/guardrails/v0.23.0/llms.txt > - v0.22.0: https://docs.nvidia.com/nemo/guardrails/v0.22.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/guardrails/_mcp/server. # Metric Reference > Reference IORails and canonical outbound HTTP metrics, including instrument types, units, labels, and emission semantics. This page lists the metrics emitted by the IORails engine and the instrumented canonical outbound HTTP client. Metrics fall into three families: * **Request-level metrics** (`guardrails.*`) describe IORails request flow: volume, errors, blocks, latency, and saturation of the streaming and non-streaming admission paths. They cover `generate_async()`, `stream_async()`, and the rails-only [`check_async()`](/run-guardrailed-inference/using-python-apis/check-messages), which shares the non-streaming admission path. * **LLM client-side metrics** (`gen_ai.client.*`) describe downstream LLM calls IORails issues. These follow the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-metrics/) and use the bucket boundaries recommended by that spec. * **HTTP client-side metrics** (`http.client.*`) describe outbound requests sent through an explicitly instrumented canonical HTTP client. IORails request and LLM metrics require `metrics.enabled: true` and a configured `MeterProvider`. HTTP client metrics require a configured `MeterProvider` and `instrument_http_client(..., metrics_enabled=True)`. ## Request-Level Metrics | Metric | Instrument | Unit | Labels | Description | | ----------------------------- | ------------- | ---- | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | | `guardrails.requests` | Counter | `1` | — | Total IORails requests handled, incremented on entry. | | `guardrails.requests.errors` | Counter | `1` | `error.type` | Requests that ended in an unhandled error. `error.type` is the exception class name (for example `NonStreamingWorkQueueFullError`, `TimeoutError`). | | `guardrails.requests.blocked` | Counter | `1` | `rail.type` | Requests blocked by an input or output rail. `rail.type` is `Input` or `Output`. | | `guardrails.request.duration` | Histogram | `s` | — | End-to-end request duration. For non-streaming requests this includes queue-wait time. | | `guardrails.requests.active` | UpDownCounter | `1` | — | Requests currently in flight. Covers both streaming and non-streaming. | ### Bucket Boundaries: `guardrails.request.duration` The duration histogram buckets use seconds: ```text [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1.0, 2.5, 5.0, 7.5, 10.0] ``` ## Saturation Metrics These metrics expose the internal admission paths so you can detect overload before users encounter errors. ### Non-Streaming Path (Admission Queue) | Metric | Instrument | Unit | Description | | --------------------------------- | --------------- | ---- | ---------------------------------------------------------------------------------------------------------------- | | `guardrails.nonstream.queued` | ObservableGauge | `1` | Requests buffered in the admission queue, not yet picked up by a worker. | | `guardrails.nonstream.active` | ObservableGauge | `1` | Requests currently executing on a worker. | | `guardrails.nonstream.rejections` | Counter | `1` | Submissions rejected with `NonStreamingWorkQueueFullError` because the admission queue exceeded its depth limit. | `queued` and `active` are read live at collection time, so dashboards always show the current state. After `IORails.stop()` is called, both gauges return no observations rather than stale values. ### Streaming Path (Concurrency Semaphore) | Metric | Instrument | Unit | Description | | ------------------------------ | ------------- | ---- | --------------------------------------------------------------------------------------------------------------------------------- | | `guardrails.stream.active` | UpDownCounter | `1` | In-progress streaming requests holding a semaphore permit. | | `guardrails.stream.rejections` | Counter | `1` | Streaming requests rejected with `StreamingCapacityExceededError` because the streaming concurrency semaphore was fully occupied. | ### Cross-Checking Saturation Metrics At any collection instant, the sum of the per-path saturation gauges should approximately equal `guardrails.requests.active`: ```text guardrails.requests.active ≈ guardrails.nonstream.queued + guardrails.nonstream.active + guardrails.stream.active ``` A persistent drift between the two is a useful integrity check during dashboard development. ### Dual-Counted Rejections A `NonStreamingWorkQueueFullError` rejection on the non-streaming path increments **both**: * `guardrails.nonstream.rejections` (saturation signal) * `guardrails.requests.errors{error.type=NonStreamingWorkQueueFullError}` (error signal) The streaming path behaves the same way. A `StreamingCapacityExceededError` rejection increments **both**: * `guardrails.stream.rejections` (saturation signal) * `guardrails.requests.errors{error.type=StreamingCapacityExceededError}` (error signal) This is intentional: dashboards built around either signal alone still reflect the rejection. Because `error.type` carries the exception class name, the two overload paths stay distinguishable on the error signal alone. ## HTTP Client Metrics The canonical outbound HTTP instrumentation records one observation for each logical request, including all retry attempts performed by a wrapped retrying client. | Metric | Instrument | Unit | Labels | Description | | ------------------------------ | ---------- | ---- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | | `http.client.request.duration` | Histogram | `s` | `http.request.method`, `server.address`, `server.port`, optionally `http.response.status_code` and `error.type` | Wall-clock duration of an outbound HTTP request. | The instrumentation normalizes the method to uppercase. The server port is the explicit URL port or the default port for `http` and `https`. A response contributes `http.response.status_code`. Status codes of 400 or greater also contribute `error.type` as the status string. A raised exception contributes its class name as `error.type` and has no response-status label. The metric does not include URL paths, query parameters, headers, bodies, credentials, or exception messages. URLs without a host or a recognized port do not produce an observation. ### Bucket Boundaries: `http.client.request.duration` The histogram uses the OpenTelemetry HTTP client recommendation in seconds: ```text [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1.0, 2.5, 5.0, 7.5, 10.0] ``` Use [`instrument_http_client`](/configure-guardrails/actions/outbound-http#add-privacy-safe-instrumentation) with `metrics_enabled=True` to enable this metric. `create_http_client` does not enable it automatically. ## LLM Client-Side Metrics These metrics are recorded once per downstream LLM call, not once per IORails request, and follow the OpenTelemetry GenAI semantic conventions. | Metric | Instrument | Unit | Labels | Description | | ----------------------------------------------- | ---------- | --------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `gen_ai.client.token.usage` | Histogram | `{token}` | `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.token.type` | Number of tokens consumed by an LLM call. Each call records two observations distinguished by the required `gen_ai.token.type` label (`input` or `output`). | | `gen_ai.client.operation.duration` | Histogram | `s` | `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model`, optionally `error.type` | Wall-clock duration of one LLM call. `error.type` is added as a conditional label only when the call raises. | | `gen_ai.client.operation.time_to_first_chunk` | Histogram | `s` | `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model` | Streaming-only. Time from request issue to the first content-bearing chunk. | | `gen_ai.client.operation.time_per_output_chunk` | Histogram | `s` | `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model` | Streaming-only. Inter-chunk gap; one observation per content-bearing chunk after the first. | > **Note** > > `gen_ai.token.type` only takes the values `input` and `output` per spec. > Reasoning and cached tokens are exposed as **span attributes** (`gen_ai.usage.reasoning.output_tokens` and so on), not as additional metric label values. ### Bucket Boundaries Per the OpenTelemetry GenAI spec, durations use powers-of-two boundaries up to \~82 s: ```text [0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56, 5.12, 10.24, 20.48, 40.96, 81.92] ``` Token counts use powers-of-four boundaries up to \~67M tokens: ```text [1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864] ``` Both match the spec exactly so backends auto-render the distributions correctly. ### Streaming vs. Non-Streaming Emission | Metric | Non-streaming | Streaming | | ----------------------------------------------- | :-----------: | :------------------------------------------: | | `gen_ai.client.token.usage` | ✓ | ✓ (when the upstream provider returns usage) | | `gen_ai.client.operation.duration` | ✓ | ✓ | | `gen_ai.client.operation.time_to_first_chunk` | — | ✓ | | `gen_ai.client.operation.time_per_output_chunk` | — | ✓ | For streaming responses, `token.usage` is emitted only when the upstream provider returns a `usage` field. This is common when `stream_options.include_usage=true` is forwarded. When usage is absent, no observation is recorded; "no observation" is deliberately distinct from "0 tokens". ## Common Label Reference | Label | Used On | Values | Notes | | --------------------------- | ------------------------------------ | --------------------------------------------- | ------------------------------------------------------------------------------------------------------- | | `error.type` | Request, LLM, and HTTP error metrics | Exception class name or HTTP error status | For example `NonStreamingWorkQueueFullError`, `TimeoutError`, `HTTPConnectionError`, or `503`. | | `rail.type` | `guardrails.requests.blocked` | `Input`, `Output` | Identifies whether an input or output rail blocked the request. Matches the `rail.type` span attribute. | | `gen_ai.operation.name` | All `gen_ai.client.*` | For example `chat`, `completion`, `embedding` | OpenTelemetry GenAI operation name. | | `gen_ai.provider.name` | All `gen_ai.client.*` | For example `openai`, `anthropic` | OpenTelemetry GenAI provider name. | | `gen_ai.request.model` | All `gen_ai.client.*` | For example `gpt-4o-mini` | The model name passed in the request. | | `gen_ai.token.type` | `gen_ai.client.token.usage` | `input`, `output` | Required label per spec. | | `http.request.method` | `http.client.request.duration` | Uppercase HTTP method | For example `GET` or `POST`. | | `server.address` | `http.client.request.duration` | Destination host | Excludes credentials and port. | | `server.port` | `http.client.request.duration` | Destination port | Uses `80` or `443` when omitted from an HTTP or HTTPS URL. | | `http.response.status_code` | `http.client.request.duration` | HTTP status code | Present only when a response is received. | ## Public API Stability The metric names listed on this page are part of the library's public API, so dashboards and alerts can reference them. The library tests assert on the raw strings for this reason. Bucket boundaries follow the relevant OpenTelemetry HTTP or GenAI specification and can change if those specifications change. ## Related Resources * [Enable Guardrails Metrics](/observability/metrics/enable-metrics) — Minimal SDK setup with console output. * [OpenTelemetry Metrics Integration](/observability/metrics/opentelemetry-integration) — Production exporters: OTLP, Prometheus. * [Outbound HTTP in Actions](/configure-guardrails/actions/outbound-http): Canonical client ownership, retries, instrumentation, and testing. * [OpenTelemetry HTTP metrics specification](https://opentelemetry.io/docs/specs/semconv/http/http-metrics/): Upstream HTTP client metric conventions. * [OpenTelemetry GenAI metrics specification](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-metrics/) — Upstream semantic conventions.