Observability for NVIDIA NIM for Nemotron Parse#

Use this documentation to monitor NVIDIA NIM for Nemotron Parse.

Prometheus Metrics#

Nemotron Parse NIM exposes metrics in Prometheus text format at /v1/metrics on the HTTP listen address. With the default NIM_SERVER_BIND_ADDR, the endpoint is http://localhost:8000/v1/metrics.

You can verify the endpoint with the following command:

curl http://localhost:8000/v1/metrics

Configure Prometheus to scrape the endpoint. For example:

scrape_configs:
  - job_name: parse-nim
    metrics_path: /v1/metrics
    static_configs:
      - targets: ["<nim-host>:8000"]

Nemotron Parse NIM doesn’t support OpenTelemetry. Use the Prometheus metrics endpoint and the logs to monitor the NIM.

Request Counters#

Metric	Type	Description
`parse_requests_total`	counter	Parse inference requests received by the HTTP and gRPC handlers.
`parse_errors_total`	counter	Requests counted by the inference handlers that return an error.

These counters have no labels, combine both transports, and reset when the server restarts. They include input validation errors that reach an inference handler. Health checks, control routes, unknown HTTP paths, unsupported HTTP methods, and gRPC requests with an unknown model name do not increment them.

The endpoint does not provide a Parse latency histogram or a pages-per-second metric. Use workload measurements to evaluate latency and throughput.

System Metrics#

The endpoint also samples host, process, and GPU metrics. A metric is omitted when its measurement is unavailable. Host and process metrics have no labels. GPU metrics have gpu, uuid, and name labels; an unavailable UUID or name is an empty string.

Metric	Type	Unit
`host_cpu_logical_cores`	gauge	Logical CPU cores
`host_memory_total_bytes`	gauge	Bytes
`host_memory_available_bytes`	gauge	Bytes
`host_memory_cgroup_limit_bytes`	gauge	Bytes
`process_virtual_memory_bytes`	gauge	Bytes
`process_resident_memory_bytes`	gauge	Bytes
`process_cpu_seconds_total`	counter	CPU seconds
`process_start_time_seconds`	gauge	Seconds since the Unix epoch
`process_open_fds`	gauge	File descriptors
`process_max_fds`	gauge	File descriptors
`gpu_memory_total_bytes`	gauge	Bytes
`gpu_memory_used_bytes`	gauge	Bytes
`gpu_memory_free_bytes`	gauge	Bytes
`gpu_utilization_ratio`	gauge	Fraction from 0 to 1
`gpu_memory_utilization_ratio`	gauge	Fraction from 0 to 1
`gpu_power_usage_watts`	gauge	Watts
`gpu_power_limit_watts`	gauge	Watts
`gpu_temperature_celsius`	gauge	Degrees Celsius
`gpu_sm_clock_mhz`	gauge	MHz
`gpu_memory_clock_mhz`	gauge	MHz
`gpu_total_energy_consumption_joules`	gauge	Joules

GPU memory gauges describe the device as a whole. The process memory logs described below measure the current NIM process instead.

Logs#

Use RUST_LOG to set the Rust tracing filter and LOG_FORMAT to select pretty, json, or compact output. For details, see Environment Variables for NVIDIA NIM for Nemotron Parse.

To follow logs from a Docker container, run:

docker logs <container-id> -f

To follow logs from a pod on Kubernetes, run:

kubectl logs -n <namespace> <pod-name> -f

Process GPU Memory#

Set NIM_PIPELINE_VRAM_TELEMETRY=1 and enable DEBUG logging to report process GPU-memory usage after each decoder iteration:

-e NIM_PIPELINE_VRAM_TELEMETRY=1 -e RUST_LOG=debug -e LOG_FORMAT=json

Each [VRAM:iteration] record includes the iteration, active decoder slots, integer MiB (used_mb), and the change from the previous sample (delta_mb). The measurement sums allocations for the current process across its unique participating GPU UUIDs, including driver and context memory. Missing measurements and deltas are reported as unavailable.

Both the telemetry setting and DEBUG logging are required for iteration measurements. Startup [VRAM:startup] checkpoints are emitted separately, even when this setting is off. Process startup checkpoints write plain text to stderr regardless of RUST_LOG or LOG_FORMAT.

Disable iteration telemetry when measuring performance because the memory queries add diagnostic work.