Connect to Datadog
Use the shared capture and verification path to make runs captured by NVIDIA NeMo Relay available to a team investigating traces in Datadog. The example routes Relay OTLP traces to an organization-operated OpenTelemetry Collector; that Collector forwards them to your Datadog site.
Select --destination datadog in the parent guide’s example. Supply your running Collector’s trace URL. Its Datadog exporter and credentials stay in that collector or tracing backend.
Find service relay-support-triage and apply this page’s destination checks.
The example uses OpenInference to keep
the same agent/model/tool roles as the Arize Phoenix
and LangSmith recipes.
Datadog accepts both OpenInference and OpenTelemetry GenAI
conventions; use gen_ai when your pipeline standardizes on those attributes.
This recipe verifies APM trace arrival. Enabling Datadog’s Agent/LLM
Observability product requires its additional
OpenTelemetry configuration.
Choose the collector or tracing backend
Datadog recommends an Agent or OpenTelemetry Collector for production OTLP traffic. The collector or tracing backend can add host metadata, process spans, and apply central sampling. Choose one that the Relay worker or application can reach and that your platform team already operates. A direct Datadog OTLP trace intake is available when an Agent or Collector is not feasible; it uses a site-specific HTTPS endpoint and an API-key header.
The following plugins.toml example sends OpenInference traces to a managed
Collector. Replace the illustrative host with its actual HTTPS trace URL:
If the Collector requires authentication from Relay, add its approved header
with header_env; keep the secret outside plugins.toml. The Collector owns
the Datadog site and API key in this topology. Follow Datadog’s
Collector setup
or Agent OTLP ingestion
for the chosen collector or tracing backend.
Check Datadog APM
Verification is manual in Datadog; the companion’s --verify option supports
Phoenix only. Confirm intake in Datadog as well as successful forwarding by your collector.
Find a recent trace under the service relay-support-triage. Confirm that
its agent span has the observed model and tool work beneath it, then inspect
model identity and sanitized payload fields.
Collector processing can change operation names. Compare the resulting APM trace with Relay’s local events, accounting for Collector mapping when matching agent, model, and tool activity.
Direct intake does not automatically compute APM trace metrics; plan that separately if the enterprise dashboard needs them. Datadog’s direct intake guide explains that option and its required headers.