> This page is for version 1.0.0.
> For other versions, use one of these documentation indexes:
> - 1.10.1 (default): https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-contentsafety/1.10.1/llms.txt
> - 1.0.0: https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-contentsafety/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-contentsafety/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-contentsafety/_mcp/server.

# Observability

> Llama 3.1 NemoGuard 8B ContentSafety NIM supports exporting metrics and traces in an OpenTelemetry-compatible format. Additionally, the microservice and its underlying NVIDIA Triton Inference Server e

## About Observability

Llama 3.1 NemoGuard 8B ContentSafety NIM supports exporting metrics and traces in an [OpenTelemetry](https://opentelemetry.io/)-compatible format.
Additionally, the microservice and its underlying NVIDIA [Triton Inference Server](https://developer.nvidia.com/triton-inference-server) expose metrics through [Prometheus](https://prometheus.io/) endpoints.

To collect these metrics and traces, export them to a running [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) instance, which can then export them to any OTLP-compatible backend.

## Metrics and Traces

You can collect metrics from both the NIM microservice and the Triton Inference Server instance.

The following environment variables are related to exporting OpenTelemetry metrics and traces from the NIM microservice.

| Variable                                              | Description                                                                                |
| ----------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `OTEL_SERVICE_NAME`                                   | Specifies the name of the service to use in the exported metrics.                          |
| `OTEL_EXPORTER_OTLP_ENDPOINT`                         | Specifies the endpoint of an OTLP gRPC receiver.                                           |
| `OTEL_METRICS_EXPORTER="otlp"`                        | Specifies to export metrics to the specified `OTEL_EXPORTER_OTLP_ENDPOINT` in OTLP format. |
| By default, metrics are printed to the container log. |                                                                                            |
| `OTEL_TRACES_EXPORTER="otlp"`                         | Specifies to export traces to the specified `OTEL_EXPORTER_OTLP_ENDPOINT` in OTLP format.  |
| By default, traces are printed to the container log.  |                                                                                            |

The NIM microservice and Triton Inference Server also expose metrics in Prometheus format.
You can access these metrics through the NIM microservice API at `<nim-host>:8000/v1/metrics`, and the Triton metrics endpoint at `<nim-host>:8002/metrics`, respectively.

## Enabling OpenTelemetry

The following example requires that an OpenTelemetry Collector gRPC receiver is running at `<opentelemetry-collector-host>` on port `<opentelemetry-collector-grpc-port>`.

```bash
export IMG_NAME=nvcr.io/nim/nvidia/llama-3.1-nemoguard-8b-content-safety
export IMG_TAG=1.0.0

# Choose a container name for bookkeeping
export CONTAINER_NAME=$(basename $IMG_NAME)

# Set the OTEL environment variables to enable metrics exporting
export OTEL_SERVICE_NAME=$CONTAINER_NAME
export OTEL_METRICS_EXPORTER=otlp
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT="http://<opentelemetry-collector-host>:<opentelemetry-collector-grpc-port>"

docker run --runtime=nvidia -d --name=$CONTAINER_NAME \
  ... \
  -e OTEL_SERVICE_NAME \
  -e OTEL_METRICS_EXPORTER \
  -e OTEL_TRACES_EXPORTER \
  -e OTEL_EXPORTER_OTLP_ENDPOINT \
  ... \
  $IMG_NAME:$IMG_TAG
```

## Receiving and Exporting Telemetry Data

The following OpenTelemetry Collector configuration enables both metrics and tracing exports.

Two receivers are defined:

* An OTLP receiver that receives both metrics and trace data from the NIM microservice.

* A Prometheus receiver scrapes Triton Inference Server metrics.

Three exporters are defined:

* A Zipkin exporter that exports to a running Zipkin instance.

* An OTLP gRPC exporter that exports to a downstream collector or backend, such as Datadog.

* A debug exporter that prints received data to the console.
  This exporter is helpful for testing and development purposes.

Traces are received exclusively by the OTLP receiver and exported by both the Zipkin and debug exporters.
Metrics are received by the OTLP and Prometheus receivers.
The metrics are exported by the OTLP and debug exporters.

```yaml
receivers:
  otlp:
    protocols:
      grpc:
      http:
        cors:
          allowed_origins:
            - "*"
  prometheus:
    config:
      scrape_configs:
        - job_name: nim-triton-metrics
          scrape_interval: 10s
          static_configs:
            - targets: ["<nim-endpoint>:8002"]
exporters:
  # NOTE: Prior to v0.86.0 use `logging` instead of `debug`.
  zipkin:
    endpoint: "<zipkin-endpoint>:<zipkin-port>/api/v2/spans"
  otlp:
    endpoint: "<otlp-metrics-endpoint>:<otlp-metrics-port>"
    tls:
      insecure: true
  debug:
    verbosity: detailed
service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug, zipkin]
    metrics:
      receivers: [otlp, prometheus]
      exporters: [debug, otlp]
```

## Related Information

* [OpenTelemetry Collector Installation](https://opentelemetry.io/docs/collector/installation/).
  If deployed correctly, metrics and traces are visible in the collector’s console logs.

* Reference Documentation

  * [OpenTelemetry Environment Variables](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/)

  * [OTLP Exporter Configuration](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/)

  * [OTLP Receiver](https://github.com/open-telemetry/opentelemetry-collector/blob/main/receiver/otlpreceiver/README.md)

  * [Prometheus Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/prometheusreceiver/README.md)

  * [Zipkin Exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/zipkinexporter/README.md)

  * [OTLP gRPC Exporter](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlpexporter/README.md)

  * [Debug Exporter](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/debugexporter/README.md)

* [OTLP Ingestion by the Datadog Agent](https://docs.datadoghq.com/opentelemetry/interoperability/otlp_ingest_in_the_agent/?tab=host) describes how to use observability features in the Datadog Agent.