> This page is for NeMo Relay, version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v0.11.2) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.11.2: https://docs.nvidia.com/nemo/relay/v0.11.2/llms.txt
> - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.

# Connect to Datadog

> Send Relay traces through an enterprise OTLP collector or tracing backend to Datadog.

Use the [shared capture and verification path](/nemo/relay/dev/user-guides/connecting-to-observability-platforms)
to make runs captured by [NVIDIA NeMo Relay](https://github.com/NVIDIA/NeMo-Relay)
available to a team investigating traces in [Datadog](https://www.datadoghq.com/).
The example routes Relay OTLP traces to an organization-operated
[OpenTelemetry Collector](https://opentelemetry.io/); that Collector forwards
them to your Datadog site.

> **Use the same companion workload**
>
> Select `--destination datadog` in the parent guide's example. Supply your running Collector's trace URL. Its Datadog exporter and credentials stay in that collector or tracing backend.
> Find service `relay-support-triage` and apply this page's destination checks.

The example uses [OpenInference](https://github.com/Arize-ai/openinference) to keep
the same agent/model/tool roles as the [Arize Phoenix](https://arize.com/phoenix/)
and [LangSmith](https://www.langchain.com/langsmith/observability) recipes.
Datadog accepts both OpenInference and OpenTelemetry GenAI
conventions; use `gen_ai` when your pipeline standardizes on those attributes.
This recipe verifies **APM trace arrival**. Enabling Datadog's Agent/LLM
Observability product requires its additional
[OpenTelemetry configuration](https://docs.datadoghq.com/llm_observability/instrument/otel_instrumentation/).

## Choose the collector or tracing backend

Datadog recommends an Agent or OpenTelemetry Collector for production OTLP
traffic. The collector or tracing backend can add host metadata, process spans, and apply
central sampling. Choose one that the Relay worker or application can reach
and that your platform team already operates. A direct Datadog
[OTLP trace intake](https://docs.datadoghq.com/opentelemetry/setup/otlp_ingest/traces/)
is available when an Agent or Collector is not feasible; it uses a
site-specific HTTPS endpoint and an API-key header.

The following `plugins.toml` example sends OpenInference traces to a managed
Collector. Replace the illustrative host with its actual HTTPS trace URL:

```toml
version = 1

[[components]]
kind = "observability"
enabled = true

[components.config]
version = 4

[components.config.opentelemetry]
enabled = true

[[components.config.opentelemetry.endpoints]]
type = "openinference"
endpoint = "https://otel-gateway.example.com/v1/traces"
transport = "http_binary"
service_name = "relay-support-triage"
```

If the Collector requires authentication from Relay, add its approved header
with `header_env`; keep the secret outside `plugins.toml`. The Collector owns
the Datadog site and API key in this topology. Follow Datadog's
[Collector setup](https://docs.datadoghq.com/opentelemetry/setup/collector_exporter/)
or [Agent OTLP ingestion](https://docs.datadoghq.com/opentelemetry/setup/otlp_ingest_in_the_agent/)
for the chosen collector or tracing backend.

## Check Datadog APM

Verification is manual in Datadog; the companion's `--verify` option supports
Phoenix only. Confirm intake in Datadog as well as successful forwarding by your collector.

Find a recent trace under the service `relay-support-triage`. Confirm that
its agent span has the observed model and tool work beneath it, then inspect
model identity and sanitized payload fields.

Collector processing can change operation names. Compare the resulting APM
trace with Relay's local events, accounting for Collector mapping when matching
agent, model, and tool activity.

| Missing evidence           | First place to check                                                                            |
| -------------------------- | ----------------------------------------------------------------------------------------------- |
| No trace                   | Relay exporter delivery, Collector or Agent ingestion, and the Datadog site.                    |
| Partial trace              | Relay capture coverage and any sampling or span processing in the collector or tracing backend. |
| Unexpected operation names | The selected trace projection and Datadog's OTLP span mapping.                                  |

Direct intake does not automatically compute APM trace metrics; plan that
separately if the enterprise dashboard needs them. Datadog's
[direct intake guide](https://docs.datadoghq.com/opentelemetry/setup/otlp_ingest/traces/)
explains that option and its required headers.