Observability#

NeMo RL is instrumented with OpenTelemetry via the nemo-lens library. It emits traces at RL-algorithm boundaries (rollout, generation, reward, advantage, policy update, …) and metrics for async efficiency accounting and vLLM generation.

Telemetry exports OTLP and works with any OTLP-compatible backend or an OpenTelemetry Collector (e.g. Jaeger, Grafana Tempo, or an OpenTelemetry Collector that fans out to your backend of choice).

Telemetry is off by default. nemo-lens ships as a base dependency, so it reaches every worker venv, and telemetry.enabled is the single switch: while it is false every instrumentation site is a ~0-cost no-op.

What’s in this section#

Scope#

This documentation covers NeMo-RL-specific usage: the telemetry: config block, RL span names, rl.* metric names, and the two-layer vLLM tracing integration.

For general concepts — the span-group mechanism, instrumentation primitives, the configuration model, custom exporters, resource detection — see the lens documentation. This section links to lens docs when relevant rather than duplicating them.

Concern

Owned by

telemetry: YAML block

NeMo-RL (this section)

RLSpanGroup groups + presets, rl.* span/metric names

NeMo-RL (this section)

Driver/worker telemetry lifecycle, vLLM two-layer tracing

NeMo-RL (this section)

managed_span / trace_fn / span_cm, config model, exporters, resource detection

lens

Install#

Nothing to install: nemo-lens[sdk,aiohttp] is a base dependency, so a normal uv sync covers the driver and every worker venv.

Quick start#

Add a telemetry: block to your run config:

telemetry:
  enabled: true
  span_groups: default   # coarse-grained; safe for production

Point it at a backend and run:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317   # your OTLP backend / collector

uv run examples/run_grpo.py --config examples/configs/grpo_math_1B.yaml

With default span groups, NeMo-RL emits a handful of coarse spans (startup, job, checkpoint, evaluate, model load) plus whatever rl.* metrics the driver’s logger produces. Switch to per_step for per-step traces (rollout/generation/reward/…), or all for everything.

Keeping the settings in the config file is what makes a run’s telemetry reproducible from the file alone. The endpoint is the exception: OTEL_EXPORTER_OTLP_* are the standard OpenTelemetry variables, and they belong in the environment because they describe where you are running, not what you are measuring. See Configuration.

What gets instrumented#

The instrumented entry points — those in the table below, plus examples/run_sft_v2.py — call init_telemetry_driver(config, algorithm="<algo>") before init_ray() (so the resolved NEMO_RL_OTEL_* settings are snapshotted into the Ray runtime_env and inherited by every worker) and shutdown_telemetry() from a finally block wrapping the whole run, so buffered spans are flushed even when the run fails.

Not every entry point is wired up: run_eval.py, run_grpo_sliding_puzzle.py, run_vlm_grpo.py, run_vlm_mpo.py, run_vlm_sft.py and run_xtoken_off_policy_distillation.py emit nothing today. A run launched through one of those produces no spans and no metrics, whatever telemetry.enabled says.

Algorithm

Entry point

Representative spans

GRPO (sync + async)

examples/run_grpo.py

rl.grpo.step, rl.grpo.generation, rl.grpo.reward_calculation, rl.grpo.policy_and_reference_logprobs, rl.grpo.advantage_calculation, rl.grpo.policy_training

PPO

examples/run_ppo.py

rl.ppo.step, rl.ppo.generation, rl.ppo.reward_calculation, rl.ppo.advantage_calculation, rl.ppo.policy_training, rl.ppo.value_training

SFT

examples/run_sft.py

rl.sft.step, rl.sft.data_processing, rl.sft.policy_training

DPO

examples/run_dpo.py

rl.dpo.step, rl.dpo.policy_training

RM

examples/run_rm.py

rl.rm.step

Distillation

examples/run_distillation.py

rl.distillation.step, rl.distillation.generation, rl.distillation.teacher_logprob_inference, rl.distillation.policy_training

SingleController (GRPO + PPO)

examples/run_grpo_single_controller.py

rl.sc.step, rl.sc.generate_and_push, rl.sc.policy_and_reference_logprobs, rl.sc.advantage_calculation, rl.sc.policy_training, rl.data_plane.*

vLLM generation

nemo_rl/models/generation/vllm/vllm_generation.py

rl.vllm.generate, rl.vllm.generate_text

The SingleController path differs in where the spans come from. Its driver only builds resources and launches SingleControllerActor, so the actor opens the job and step spans and flushes its own telemetry; the entrypoint still calls init_telemetry_driver before init_ray(), because that is what puts the resolved settings in the environment the actor inherits.

Each span belongs to a span group that controls whether it is emitted at runtime. See Span Groups for the full per-algorithm span table.

What gets exported#

  • Traces: any OTLP-compatible backend (Jaeger, Grafana Tempo, an OpenTelemetry Collector, …) via OTLP.

  • Metrics: the rl.efficiency.* async accounting teed from the driver’s metrics logger, the training scalars (rl.reward.mean, rl.policy.loss, …), rl.setup.duration for the startup phases, the rl.vllm.* engine deltas, the driver-side gen_ai.* generation series, and rl.host.memory.* per node — see Metrics.

  • Logs (optional): via the OTel log bridge when telemetry.logs_enabled is true — correlates Python logging records with the active span’s trace ID.

Every process that enables telemetry exports, each labelled with nv.dl.rank / nv.dl.world_size. Narrowing a large fleet down is a collector-side filter rather than a config setting — see Configuration — Which ranks export.