> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Local Resource Monitor (Local)

This tool is **local-only**. [`dynamo_local_resource_monitor.py`](https://github.com/ai-dynamo/dynamo/blob/main/dev/observability/dynamo_local_resource_monitor.py) runs on the host machine (not inside a container) and attributes GPU/CPU/IO usage to individual Dynamo processes by PID. It is not part of a Kubernetes deployment. For the setup walkthrough and design rationale, see the [Local Resource Monitor guide](local-resource-monitor-guide.md).

The monitor samples per-process resource usage (VRAM, GPU utilization, PCIe bandwidth, CPU, disk I/O, network I/O) for Dynamo inference processes, labeled by GPU, PID, and process name. It always exposes Prometheus metrics at `/metrics`; when the dashboard dependencies are installed, the same endpoint also serves a WebSocket dashboard at `/`.

Run it on the host:

```bash
pip install psutil nvidia-ml-py prometheus-client
python3 dev/observability/dynamo_local_resource_monitor.py --host 0.0.0.0 --port 8051
```

## Metrics

All metrics use the `dynamo_` prefix. Unlike the runtime's `dynamo_component_*` families, these are host-side gauges labeled by physical GPU and OS process.

### GPU metrics

**`dynamo_gpu_memory_used_gib`** `gauge`

GPU memory used by a process, in GiB. Labeled by `gpu`, `pid`, and `process_name`.

---

**`dynamo_gpu_memory_total_gib`** `gauge`

Total GPU memory, in GiB. Labeled by `gpu`.

---

**`dynamo_gpu_utilization_percent`** `gauge`

GPU utilization, as a percentage. Labeled by `gpu`.

---

**`dynamo_gpu_temperature_celsius`** `gauge`

GPU temperature, in degrees Celsius. Labeled by `gpu`.

---

**`dynamo_gpu_pcie_tx_gbps`** `gauge`

GPU PCIe transmit throughput, in GB/s. Labeled by `gpu`.

---

**`dynamo_gpu_pcie_rx_gbps`** `gauge`

GPU PCIe receive throughput, in GB/s. Labeled by `gpu`.

---

### CPU metrics

**`dynamo_cpu_utilization_percent`** `gauge`

Overall CPU utilization, as a percentage.

---

**`dynamo_cpu_process_percent`** `gauge`

CPU usage by process-name group, as a percentage. Labeled by `process_name`.

---

### Network metrics

**`dynamo_network_sent_mbps`** `gauge`

Network bytes sent, in MB/s.

---

**`dynamo_network_recv_mbps`** `gauge`

Network bytes received, in MB/s.

---

### Disk metrics

**`dynamo_disk_read_mbps`** `gauge`

Disk read throughput, in MB/s.

---

**`dynamo_disk_write_mbps`** `gauge`

Disk write throughput, in MB/s.

---

## CLI flags

**`--port`** `integer` — default: 8051

Port for the `/metrics` exporter (and the dashboard at `/` when dependencies are installed).

---

**`--host`** `string` — default: 0.0.0.0

Host address to bind.

---

**`--main-interval`** `integer (ms)` — default: 200

Main collection interval in milliseconds, covering CPU, GPU memory/utilization/temperature, and network (5 samples/s).

---

**`--disk-interval`** `integer (ms)` — default: 1000

Disk I/O collection interval in milliseconds (1 sample/s).

---

**`--window`** `integer (s)` — default: 900

Dashboard rolling window in seconds (default 900 = 15 minutes).

---

**`--top-n`** `integer` — default: 12

Dashboard process-series limit per chart.

---

## Related

* [Local Resource Monitor guide](local-resource-monitor-guide.md) — setup, Prometheus profile gating, and the WebSocket dashboard.
* [Metrics Catalog](/dynamo/dev/reference/observability/metrics-catalog) — the runtime's `dynamo_*` metrics.
* [Environment Variables](/dynamo/dev/reference/observability/environment-variables) — runtime observability configuration.