> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/v0.7.0/observability/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# Observability

> Collect rollout evidence and monitor Gym services.

NeMo Gym provides two independent observability features:

| Feature          | Use it to                                                           | Enable with                                          |
| ---------------- | ------------------------------------------------------------------- | ---------------------------------------------------- |
| Rollout evidence | Inspect model calls, agent activity, and saved trajectory health    | `observability_enabled` and `model_call_capture_dir` |
| OpenTelemetry    | Trace requests across services and monitor runtime metrics and logs | `telemetry.enabled` or `NEMO_GYM_OTEL_ENABLED=1`     |

Each feature has its own configuration. You can enable either or both.

## Get Started

#### [Rollout Evidence](/main/observability/rollout-evidence)

Collect and inspect model-call capture, agent observations, and normalized trajectories.

#### [Collector for Submitted Jobs](/main/observability/eval-submit-collector)

Run an OpenTelemetry Collector beside every `gym eval submit` job: engine, GPU and OTLP data with a per-run identity.

#### [OpenTelemetry](/main/observability/opentelemetry)

Export distributed traces, metrics, and correlated logs through nemo-lens.

> Collect rollout evidence and monitor Gym services.

## Docs

- [Rollout Evidence](https://docs.nvidia.com/nemo/gym/v0.7.0/observability/rollout-evidence.md): Collect, join, and inspect model-call and agent evidence.
- [OpenTelemetry](https://docs.nvidia.com/nemo/gym/v0.7.0/observability/opentelemetry.md): Trace and measure Gym services with nemo-lens.
- [Configuration](https://docs.nvidia.com/nemo/gym/v0.7.0/observability/configuration.md): 
- [Distributed Tracing](https://docs.nvidia.com/nemo/gym/v0.7.0/observability/distributed-tracing.md): 
- [Span Groups](https://docs.nvidia.com/nemo/gym/v0.7.0/observability/span-groups.md): 
- [Metrics](https://docs.nvidia.com/nemo/gym/v0.7.0/observability/metrics.md): 
- [Extending](https://docs.nvidia.com/nemo/gym/v0.7.0/observability/extending.md): 
- [Collector for Submitted Jobs](https://docs.nvidia.com/nemo/gym/v0.7.0/observability/eval-submit-collector.md): gym eval submit runs an OpenTelemetry Collector beside every benchmark job.