nemoguardrails.guardrails.telemetry | NVIDIA NeMo Guardrails Library Developer Guide

Inline OpenTelemetry instrumentation for the IORails engine.

All OpenTelemetry API imports are isolated in this module so the rest of the guardrails package never imports opentelemetry directly. When the opentelemetry-api package is not installed, the public entry points is_tracing_enabled, get_tracer, get_meter, and traced_request degrade gracefully (returning False, None, or a no-span / no-metric passthrough respectively). Lower-level helpers like request_span and trace_id_to_request_id require OTEL to be available and are only reachable through traced_request when a non-None tracer is provided.

Module Contents

Classes

Name	Description
`RequestInstruments`	Request-level OTEL instruments for the IORails engine.
`TracedRequest`	Handle yielded by `traced_request`.

Functions

Name	Description
`_cleanup_request_id`	Reset the request-ID ContextVar from a cleanup path, tolerating the
`_ensure_request_instruments`	Lazily create the request-level instruments and return them as a
`action_span`	Create a `guardrails.action` INTERNAL span for a rail action execution.
`api_call_span`	Create a CLIENT span for a non-LLM API call (e.g., jailbreak detection).
`are_metrics_enabled`	Return `True` when inline OTEL metrics should be emitted.
`get_meter`	Return a cached OpenTelemetry meter for nemo-guardrails, or `None`.
`get_tracer`	Return a cached OpenTelemetry tracer for nemo-guardrails, or `None`.
`is_tracing_enabled`	Return `True` when inline OTEL tracing should be active.
`llm_call_span`	Create a CLIENT span for an LLM call following GenAI semantic conventions.
`mark_rail_stop`	Set `rail.stop=True` on a rail span when the rail blocked the request.
`rail_span`	Create a `guardrails.rail` INTERNAL span for a single rail execution.
`record_nonstream_rejected`	Increment `guardrails.nonstream.rejections` by 1.
`record_request_blocked`	Increment `guardrails.requests.blocked` with a `rail.type` label.
`record_request_error`	Increment `guardrails.requests.errors` with an `error.type` label.
`record_span_error`	Record an exception on an OTEL span and set its status to ERROR.
`record_stream_rejected`	Increment `guardrails.stream.rejections` by 1.
`register_nonstream_saturation_gauges`	Register `guardrails.nonstream.queued` + `guardrails.nonstream.active`
`request_metrics`	Emit request-level OTEL metrics around the wrapped block.
`request_span`	Create a live `guardrails.request` SERVER span.
`set_speculative_span_attrs`	Stamp speculative-generation outcome attributes on a request span.
`stream_active_metric`	Context manager that tracks a stream as active for its full lifetime.
`trace_id_to_request_id`	Derive a human-readable request ID from the span’s OTEL trace ID.
`traced_request`	Unified request context: sets request ID, optionally creates a span

Data

_INVALID_TRACE_ID

_OTEL_AVAILABLE

_meter

_request_instruments

_tracer

log

API

class nemoguardrails.guardrails.telemetry.RequestInstruments(
    requests: opentelemetry.metrics.Counter,
    errors: opentelemetry.metrics.Counter,
    blocked: opentelemetry.metrics.Counter,
    duration: opentelemetry.metrics.Histogram,
    requests_active: opentelemetry.metrics.UpDownCounter,
    nonstream_rejections: opentelemetry.metrics.Counter,
    stream_active: opentelemetry.metrics.UpDownCounter,
    stream_rejections: opentelemetry.metrics.Counter
)

Dataclass

Request-level OTEL instruments for the IORails engine.

Field names mirror the emitted metric names (minus the guardrails. prefix). The saturation-metric group covers the full request lifecycle:

Aggregate: requests_active (guardrails.requests.active)
Non-streaming path: nonstream_rejections (guardrails.nonstream.rejections); the two gauges nonstream.queued and nonstream.active are registered separately via register_nonstream_saturation_gauges because ObservableGauges need a live queue reference.
Streaming path: stream_active (guardrails.stream.active) and stream_rejections (guardrails.stream.rejections).

blocked

Counter

duration

Histogram

errors

Counter

nonstream_rejections

Counter

requests

Counter

requests_active

UpDownCounter

stream_active

UpDownCounter

stream_rejections

Counter

class nemoguardrails.guardrails.telemetry.TracedRequest()

Bases: NamedTuple

Handle yielded by traced_request.

span is the IORails guardrails.request span when tracing is enabled, or None when it is not. request_id is always a 16-char hex string. Unpacks as (span, request_id) for callers that prefer positional access.

request_id

str

span

Optional[Span]

nemoguardrails.guardrails.telemetry._cleanup_request_id(
    token
) -> None

Reset the request-ID ContextVar from a cleanup path, tolerating the one expected ValueError.

ContextVar.reset() raises ValueError("... was created in a different Context") when called from a different asyncio Context than where .set() was called. That happens during async-generator cleanup (aclose() running in an outer task’s context) and is the only ValueError that reset_request_id raises today. Any other ValueError indicates an unexpected bug in the helper and is re-raised so callers see it.

nemoguardrails.guardrails.telemetry._ensure_request_instruments() -> typing.Optional[nemoguardrails.guardrails.telemetry.RequestInstruments]

Lazily create the request-level instruments and return them as a :class:RequestInstruments. Returns None when the OTEL API is not installed.

nemoguardrails.guardrails.telemetry.action_span(
    tracer: typing.Optional[opentelemetry.trace.Tracer],
    action_name: str
) -> typing.Generator[typing.Optional[opentelemetry.trace.Span], None, None]

Create a guardrails.action INTERNAL span for a rail action execution.

Yields the span (or None when tracer is None).

nemoguardrails.guardrails.telemetry.api_call_span(
    tracer: typing.Optional[opentelemetry.trace.Tracer],
    api_name: str
) -> typing.Generator[typing.Optional[opentelemetry.trace.Span], None, None]

Create a CLIENT span for a non-LLM API call (e.g., jailbreak detection).

Uses the api.name attribute rather than gen_ai.operation.name because these APIs are plain HTTP endpoints, not GenAI operations. http.* transport attributes can be added additively later without conflict. Yields the span (or None when tracer is None).

nemoguardrails.guardrails.telemetry.are_metrics_enabled(
    config_metrics: typing.Optional[nemoguardrails.rails.llm.config.MetricsConfig]
) -> bool

Return True when inline OTEL metrics should be emitted.

Requires the opentelemetry-api package to be installed and config.metrics.enabled to be True. Independent of :func:is_tracing_enabled — OTEL signals (traces, metrics, logs) are designed to be toggled independently so customers can, for example, run metrics-only for cost-optimized SLO dashboards without the overhead of full trace export.

nemoguardrails.guardrails.telemetry.get_meter() -> typing.Optional[opentelemetry.metrics.Meter]

Return a cached OpenTelemetry meter for nemo-guardrails, or None.

The meter is obtained via the OTEL API (not SDK), following the library instrumentation best practice. The application is responsible for configuring a MeterProvider before any metrics are recorded; without one, the API returns a no-op meter and all emissions are silently discarded.

nemoguardrails.guardrails.telemetry.get_tracer() -> typing.Optional[opentelemetry.trace.Tracer]

Return a cached OpenTelemetry tracer for nemo-guardrails, or None.

The tracer is obtained via the OTEL API (not SDK), following the library instrumentation best practice. The application is responsible for configuring a TracerProvider before any spans are created.

nemoguardrails.guardrails.telemetry.is_tracing_enabled(
    config_tracing: typing.Optional[nemoguardrails.rails.llm.config.TracingConfig]
) -> bool

Return True when inline OTEL tracing should be active.

Requires the opentelemetry-api package to be installed and config.tracing.enabled to be True. Other TracingConfig fields (adapters, span_format) are used by the LLMRails post-hoc tracing path and are ignored here.

nemoguardrails.guardrails.telemetry.llm_call_span(
    tracer: typing.Optional[opentelemetry.trace.Tracer],
    model_name: str,
    provider_name: str,
    operation_name: str = 'chat'
) -> typing.Generator[typing.Optional[opentelemetry.trace.Span], None, None]

Create a CLIENT span for an LLM call following GenAI semantic conventions.

Span name follows the OTEL pattern: "{operation_name} {model_name}".

operation_name defaults to "chat" because IORails only issues chat completions. In the future if any other non-chat LLM operations are supported, callers should pass an explicit operation_name from the OTEL GenAI semantic conventions.

Yields the span (or None when tracer is None).

nemoguardrails.guardrails.telemetry.mark_rail_stop(
    span: typing.Optional[opentelemetry.trace.Span],
    is_safe: bool
) -> None

Set rail.stop=True on a rail span when the rail blocked the request.

Safe to call with None (no-op) so callers don’t have to branch on whether a real span was produced — matches the record_span_error idiom. Only marks stop when is_safe is False; a passing rail leaves the attribute unset.

nemoguardrails.guardrails.telemetry.rail_span(
    tracer: typing.Optional[opentelemetry.trace.Tracer],
    flow: str,
    direction: nemoguardrails.guardrails.guardrails_types.RailDirection
) -> typing.Generator[typing.Optional[opentelemetry.trace.Span], None, None]

Create a guardrails.rail INTERNAL span for a single rail execution.

Yields the span (or None when tracer is None). The caller should set rail.stop on the span after execution if the rail blocked the request.

nemoguardrails.guardrails.telemetry.record_nonstream_rejected() -> None

Increment guardrails.nonstream.rejections by 1.

Called from the non-streaming path when the admission queue rejects a submission with asyncio.QueueFull (the queue’s reject_on_full behaviour, triggered when NONSTREAM_QUEUE_DEPTH is exceeded).

nemoguardrails.guardrails.telemetry.record_request_blocked(
    direction: nemoguardrails.guardrails.guardrails_types.RailDirection
) -> None

Increment guardrails.requests.blocked with a rail.type label.

Fires at the block sites in iorails.py (_do_generate for the non-streaming path, _generation_task for streaming) whenever the request returns REFUSAL_MESSAGE because an input or output rail flagged it. The counter is cumulative over the process lifetime; a per-rail grain (rail.name) will be added in split-2 alongside guardrails.rail.blocked.

No-op when the OTEL API is unavailable or instruments cannot be created.

nemoguardrails.guardrails.telemetry.record_request_error(
    exc: BaseException
) -> None

Increment guardrails.requests.errors with an error.type label.

request_metrics already bumps this counter when an exception propagates through its except branch (the non-streaming path). Streaming code paths catch-and-swallow exceptions inside _generation_task — converting them to error-payload chunks — so the counter never sees them via propagation. Those paths should call this helper explicitly so the errors counter reflects ALL failed requests, not just those whose exceptions bubble up.

No-op when the OTEL API is unavailable or instruments cannot be created.

nemoguardrails.guardrails.telemetry.record_span_error(
    span: typing.Optional[opentelemetry.trace.Span],
    exc: BaseException
) -> None

Record an exception on an OTEL span and set its status to ERROR.

Also sets the error.type attribute to the exception’s class name (per OTEL GenAI conditional-required convention). Safe to call with None (no-op). Use from every span helper’s except block and from callers that swallow exceptions before they can propagate.

nemoguardrails.guardrails.telemetry.record_stream_rejected() -> None

Increment guardrails.stream.rejections by 1.

Called from the streaming path when a request arrives while the stream concurrency semaphore is fully occupied (_stream_semaphore.locked()).

nemoguardrails.guardrails.telemetry.register_nonstream_saturation_gauges(
    queue: nemoguardrails.guardrails.async_work_queue.AsyncWorkQueue,
    is_running: typing.Callable[[], bool]
) -> None

ObservableGauges read live state at collection time, so both metrics reflect the current non-streaming queue + worker occupancy with no drift risk vs. an UpDownCounter lineage.

is_running is a zero-arg callable returning bool, deferred so each collection re-reads the current state (passing the bool directly would bake its start-time value into the closure). The callbacks return an empty observation list when it returns False — the state the flag holds after IORails.stop() flips self._running back to False. OTEL Python has no public unregister API for observable instruments, so this “no data points” fallback is the only way to stop a dead IORails instance from polluting collection.

No-op when the OTEL API is unavailable or no MeterProvider is configured.

nemoguardrails.guardrails.telemetry.request_metrics() -> typing.Generator[None, None, None]

Emit request-level OTEL metrics around the wrapped block.

Increments guardrails.requests on entry, bumps guardrails.requests.active (UpDownCounter) for the duration of the block, records guardrails.request.duration in seconds on exit, and increments guardrails.requests.errors with an error.type attribute when the block raises.

requests.active covers both non-streaming (queue-wait + execution) and streaming (semaphore hold) requests. Summing the per-path saturation metrics (nonstream.queued, nonstream.active, stream.active) should approximate this value at any collection instant.

Instruments are created lazily on first use. No-op when the OTEL API is not installed or instruments cannot be created.

nemoguardrails.guardrails.telemetry.request_span(
    tracer: opentelemetry.trace.Tracer
) -> typing.Generator[typing.Tuple[opentelemetry.trace.Span, str], None, None]

Create a live guardrails.request SERVER span.

Yields (span, request_id) where request_id is derived from the OTEL trace ID. The span is ended automatically when the block exits. If an exception propagates, the span records it and sets ERROR status before re-raising.

nemoguardrails.guardrails.telemetry.set_speculative_span_attrs(
    span: typing.Optional[opentelemetry.trace.Span],
    first_completed: str,
    first_rejector: str
) -> None

Stamp speculative-generation outcome attributes on a request span.

Records which branch of the speculative race finished first (input rails vs. main LLM generation) and which one ultimately rejected the request, on the IORails guardrails.request span. Safe to call with None (no-op) so callers don’t have to branch on whether tracing is enabled — matches the record_span_error / mark_rail_stop idiom.

nemoguardrails.guardrails.telemetry.stream_active_metric() -> typing.Generator[None, None, None]

Context manager that tracks a stream as active for its full lifetime.

+1 on enter / -1 on exit (finally) on guardrails.stream.active (UpDownCounter). No-op when metrics are unavailable. Wrap the block where the stream holds a semaphore permit.

nemoguardrails.guardrails.telemetry.trace_id_to_request_id(
    span: opentelemetry.trace.Span
) -> str

Derive a human-readable request ID from the span’s OTEL trace ID.

Returns the last REQUEST_ID_HEX_CHARS hex characters of the 128-bit trace ID (the low 64 bits, which carry the highest entropy). When the trace ID is zero (e.g. a NoOpTracerProvider is active) a random fallback is used.

nemoguardrails.guardrails.telemetry.traced_request(
    tracer: typing.Optional[opentelemetry.trace.Tracer],
    metrics_enabled: bool = False
) -> typing.Generator[nemoguardrails.guardrails.telemetry.TracedRequest, None, None]

Unified request context: sets request ID, optionally creates a span and/or emits request-level metrics.

The two signals are gated independently:

tracer is not None → a live guardrails.request SERVER span is created and the request ID is derived from its trace ID.
metrics_enabled=True → emit request-level OTEL metrics

All four combinations are valid. Metrics-only (tracer=None, metrics_enabled=True) is a supported setup for customers running cheap SLO dashboards without full trace export.

Yields a :class:TracedRequest (span, request_id). Callers that want to mark the request span ERROR from a deeply-nested scope should capture the yielded span and pass it explicitly to record_span_error — never rely on trace.get_current_span() which can return the host app’s ambient span when IORails tracing is disabled.

The request-ID ContextVar is always cleaned up on exit via :func:_cleanup_request_id, which tolerates the expected cross-context ValueError that async-generator cleanup can raise.

nemoguardrails.guardrails.telemetry._INVALID_TRACE_ID = 0

nemoguardrails.guardrails.telemetry._OTEL_AVAILABLE: bool = True

nemoguardrails.guardrails.telemetry._meter = None

nemoguardrails.guardrails.telemetry._request_instruments: Optional[RequestInstruments] = None

nemoguardrails.guardrails.telemetry._tracer = None

nemoguardrails.guardrails.telemetry.log = logging.getLogger(__name__)