OpenTelemetry

View as Markdown

NeMo Gym exports traces, metrics, and optional logs through nemo-lens. Telemetry is off by default.

A distributed trace follows a rollout across agent, model, and resources servers. Use it to find slow requests and service failures. For saved model calls and agent activity, see Rollout Evidence.

Quick Start

Install the optional dependency:

uv sync --extra dev --extra telemetry

Export to the console:

NEMO_GYM_OTEL_ENABLED=1 NEMO_GYM_OTEL_EXPORTER=console \
gym env start --resources-server example_single_tool_call --model-type vllm_model

Or send telemetry to an OTLP collector:

NEMO_GYM_OTEL_ENABLED=1 OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 \
gym env start --resources-server example_single_tool_call --model-type vllm_model
SignalContents
TracesRequest spans across agent, model, and resources servers
MetricsRollout and verification durations, verification success rate, active servers, and HTTP metrics
LogsOptional Python log records correlated with traces

Instrumentation is a no-op when disabled or when nemo-lens is absent.

Next Steps