Skip to navigation

NIM provides Prometheus metrics indicating request statistics. These metrics can be used to create dashboards with the Grafana dashboard. By default, these metrics are available at http://0.0.0.0:8000/v1/metrics .

You can use the following command to retrieve the metrics:

curl -X 'GET' 'http://0.0.0.0:8000/v1/metrics'

The following table describes the available metrics.

CategoryMetric NameDescription
GPUgpu_power_usage_wattsGPU instantaneous power, in watts
gpu_power_limit_wattsMaximum GPU power limit, in watts
gpu_total_energy_consumption_joulesGPU total energy consumption, in joules
gpu_utilizationGPU utilization rate (0.0–1.0)
gpu_memory_total_bytesTotal GPU memory, in bytes
gpu_memory_used_bytesUsed GPU memory, in bytes
Processprocess_virtual_memory_bytesVirtual memory size in bytes
process_resident_memory_bytesResident memory size in bytes
process_start_time_secondsStart time of the process since Unix epoch in seconds
process_cpu_seconds_totalTotal user and system CPU time spent in seconds
process_open_fdsNumber of open file descriptors
process_max_fdsMaximum number of open file descriptors
Pythonpython_gc_objects_collected_totalObjects collected during GC
python_gc_objects_uncollectable_totalUncollectable objects found during GC
python_gc_collections_totalNumber of times this generation was collected
python_infoPython platform information