nemo_gym.telemetry.config

Schema for the telemetry: block of a NeMo Gym config.

nemo_gym.telemetry.setup translates this block into NEMO_GYM_OTEL_* environment variables in the orchestrating CLI process before it spawns the server processes, so every server, and every Ray worker under them, inherits one consistent configuration. Raw NEMO_GYM_OTEL_* / NEMO_LENS_* / OTEL_* env vars always win over these YAML values: the translation uses os.environ.setdefault.

This module imports only pydantic. It never requires nemo-lens, so it is safe to import unconditionally from Gym’s config machinery.

Module Contents

Classes

Name	Description
`MemoryProfilingConfig`	Periodic host and logical-server memory sampling.
`TelemetryConfig`	OpenTelemetry / nemo-lens configuration for NeMo Gym.

API

class nemo_gym.telemetry.MemoryProfilingConfig()

Bases: BaseModel

Periodic host and logical-server memory sampling.

enabled

bool = False

Collect host memory and every server process tree.

interval_seconds

float = Field(default=1.0, gt=0)

Seconds between samples.

class nemo_gym.telemetry.TelemetryConfig()

Bases: BaseModel

OpenTelemetry / nemo-lens configuration for NeMo Gym.

Telemetry is optional twice over: it activates only when enabled is true and nemo-lens is installed (uv sync --extra telemetry). When either is missing every instrumentation site in nemo_gym degrades to a ~0-cost no-op and Gym behaves exactly as it does today.

enabled

bool = False

Master switch. When false, all instrumentation is a ~0-cost no-op.

event_loop_sample_interval_seconds

float = Field(default=1.0, gt=0)

Seconds between samples of each process’s event-loop lag, when metrics are on.

event_loop_task_count

bool = False

Also report how many asyncio tasks exist on each event loop, at every sample.

Off by default because counting tasks walks every task on the loop while holding it. That takes about a millisecond per sample at 16,000 tasks, which a driver at high concurrency can reach.

exporter

str = 'otlp'

Exporter backend: otlp | console. console writes one JSON object per line to stdout, which is what the local validation flow greps.

logs_enabled

bool = False

Bridge Python logging to OTel logs, exported with trace correlation.

memory_profiling

MemoryProfilingConfig = Field(default_factory=MemoryProfilingConfig)

Sample each Gym-managed server process tree and emit memory metrics.

metrics_enabled

bool = True

Emit metric instruments (the gym.* histograms/gauges plus the FastAPI instrumentor’s http.server.*).

otlp_endpoint

Optional[str] = None

OTEL_EXPORTER_OTLP_ENDPOINT for the otlp exporter, e.g. https://api.honeycomb.io. Optional: the standard env var still works and always wins (setdefault), so any OTLP-compatible backend or collector that expects the raw env var keeps working unchanged. Setting it here just means a run’s destination is committed with the rest of the config instead of exported by hand every time.

otlp_headers

Optional[str] = None

OTEL_EXPORTER_OTLP_HEADERS, e.g. x-honeycomb-team=.... Prefer ${oc.env:MY_API_KEY} interpolation for the secret portion so the raw credential does not live in a committed YAML file.

otlp_protocol

Optional[str] = None

OTEL_EXPORTER_OTLP_PROTOCOL: grpc (the OTel SDK default) or http/protobuf.

run_id

Optional[str] = None

Correlates every process of one gym env start / gym env test invocation. Generated in the supervisor and inherited by the servers when unset.

service_name

str = 'nemo-gym'

service.name reported to the backend.

Passed through the standard OTEL_SERVICE_NAME env var, not through instrument_fastapi(service_name=...) — that parameter is accepted and ignored by nemo-lens at the pinned commit (contrib/fastapi.py).

Each server process appends its own Gym server name, so a run yields nemo-gym/example_single_tool_call, nemo-gym/policy_model, and so on rather than three indistinguishable nemo-gym services. Set service_name_per_server to false to opt out.

service_name_per_server

bool = True

Suffix service_name with each server’s Gym config name.

On by default because Gym runs several processes that would otherwise share one service.name, which makes a backend’s service map useless. The unsuffixed name stays available as the nemo.gym.service_group resource attribute.

span_groups

str = 'default'

Span-group spec: a preset (default | per_rollout | all) or a comma-separated list of group names (e.g. "default,sandbox"). See GymSpanGroup. An unknown name is logged as a warning and ignored rather than rejected.

traces_enabled

bool = True

Emit trace spans.