Skip to navigation

Observability

View as MarkdownOpen in Claude

NIM provides Prometheus metrics indicating request statistics. These metrics can be used to create dashboards with the Grafana dashboard. By default, these metrics are available at http://0.0.0.0:8000/v1/metrics.

You can use the following command to retrieve the metrics:

curl -X 'GET' 'http://0.0.0.0:8000/v1/metrics'

The following table describes the available metrics.

CategoryMetric NameDescription
GPUgpu_power_usage_wattsGPU instantaneous power, in watts
gpu_power_limit_wattsMaximum GPU power limit, in watts
gpu_total_energy_consumption_joulesGPU total energy consumption, in joules
gpu_utilizationGPU utilization rate (0.0–1.0)
gpu_memory_total_bytesTotal GPU memory, in bytes
gpu_memory_used_bytesUsed GPU memory, in bytes
Processprocess_virtual_memory_bytesVirtual memory size in bytes
process_resident_memory_bytesResident memory size in bytes
process_start_time_secondsStart time of the process since Unix epoch in seconds
process_cpu_seconds_totalTotal user and system CPU time spent in seconds
process_open_fdsNumber of open file descriptors
process_max_fdsMaximum number of open file descriptors
Pythonpython_gc_objects_collected_totalObjects collected during GC
python_gc_objects_uncollectable_totalUncollectable objects found during GC
python_gc_collections_totalNumber of times this generation was collected
python_infoPython platform information