> This page is for version 4.3.
> For other versions, use one of these documentation indexes:
> - 4.6 (Latest) (default): https://docs.nvidia.com/holoscan/sdk-user-guide/4-6-latest/llms.txt
> - ltsb-2.0: https://docs.nvidia.com/holoscan/sdk-user-guide/ltsb-2-0/llms.txt
> - 4.5: https://docs.nvidia.com/holoscan/sdk-user-guide/4-5/llms.txt
> - 4.4: https://docs.nvidia.com/holoscan/sdk-user-guide/4-4/llms.txt
> - 4.3: https://docs.nvidia.com/holoscan/sdk-user-guide/4-3/llms.txt
> - Archives: https://docs.nvidia.com/holoscan/sdk-user-guide/archives/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/holoscan/sdk-user-guide/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/holoscan/sdk-user-guide/_mcp/server.

# holoscan::GPUResourceMonitor

> GPUResourceMonitor class.

`GPUResourceMonitor` class.

This class is responsible for monitoring the GPU resources. It provides the information about the GPU resources (through [`holoscan::GPUInfo`](../structs/gpuinfo)) to the [SystemResourceManager](systemresourcemanager) class.

The following `holoscan::GPUMetricFlag` flags are supported:

* `DEFAULT`: Default GPU metrics (`GPU_DEVICE_ID`)
* `GPU_DEVICE_ID`: GPU device ID (name, pci, serial, uuid)
* `GPU_UTILIZATION`: GPU utilization (gpu\_utilization, memory\_utilization)
* `MEMORY_USAGE`: GPU memory usage (memory\_total, memory\_free, memory\_used, memory\_usage)
* `POWER_LIMIT`: GPU power limit (power\_limit)
* `POWER_USAGE`: GPU power usage (power\_usage)
* `TEMPERATURE`: GPU temperature (temperature)
* `ALL`: All GPU metrics above

`index` information is always available.

This uses the NVML library to get the GPU information. If NVML library is not available (in case of iGPU), this class uses the CUDA Runtime API to get the GPU information.

The following information is not available when using the CUDA Runtime API:

* `GPU_DEVICE_ID`: `pci.pciDeviceId` and `pci.pciSubSystemId` are not available
* `GPU_UTILIZATION`: `gpu_utilization` and `memory_utilization` are not available
* `POWER_LIMIT`: `power_limit` is not available
* `POWER_USAGE`: `power_usage` is not available
* `TEMPERATURE`: `temperature` is not available

Example:

```cpp showLineNumbers={false}
#include <holoscan/gpu_resource_monitor.hpp>
```

## Example

```cpp showLineNumbers={false}
#include <holoscan/core/system/system_resource_manager.hpp>
#include <holoscan/logger/logger.hpp>

...

holoscan::GPUResourceMonitor gpu_resource_monitor;
gpu_resource_monitor.update(holoscan::GPUMetricFlag::ALL);
auto gpu_info = gpu_resource_monitor.gpu_info();
auto gpu_count = gpu_resource_monitor.num_gpus();

for (int i = 0; i < gpu_count; i++) {
  // Print GPU information (GPUInfo)
  HOLOSCAN_LOG_INFO("GPU {} is available", gpu_info[i].index);
  HOLOSCAN_LOG_INFO("GPU {} name: {}", i, gpu_info[i].name);
  HOLOSCAN_LOG_INFO("GPU {} is iGPU: {}", i, gpu_info[i].is_integrated);
  HOLOSCAN_LOG_INFO("GPU {} pci.busId: {}", i, gpu_info[i].pci.busId);
  HOLOSCAN_LOG_INFO("GPU {} pci.busIdLegacy: {}", i, gpu_info[i].pci.busIdLegacy);
  HOLOSCAN_LOG_INFO("GPU {} pci.domain: {}", i, gpu_info[i].pci.domain);
  HOLOSCAN_LOG_INFO("GPU {} pci.bus: {}", i, gpu_info[i].pci.bus);
  HOLOSCAN_LOG_INFO("GPU {} pci.device: {}", i, gpu_info[i].pci.device);
  HOLOSCAN_LOG_INFO("GPU {} pci.pciDeviceId: {:x}:{:x}",
                    i,
                    gpu_info[i].pci.pciDeviceId & 0xffff,
                    gpu_info[i].pci.pciDeviceId >> 16);
  HOLOSCAN_LOG_INFO("GPU {} pci.pciSubSystemId: {:x}:{:x}",
                    i,
                    gpu_info[i].pci.pciSubSystemId & 0xffff,
                    gpu_info[i].pci.pciSubSystemId >> 16);
  HOLOSCAN_LOG_INFO("GPU {} serial: {}", i, gpu_info[i].serial);
  HOLOSCAN_LOG_INFO("GPU {} uuid: {}", i, gpu_info[i].uuid);
  HOLOSCAN_LOG_INFO("GPU {} gpu_utilization: {}", i, gpu_info[i].gpu_utilization);
  HOLOSCAN_LOG_INFO("GPU {} memory_utilization: {}", i, gpu_info[i].memory_utilization);
  HOLOSCAN_LOG_INFO("GPU {} memory_total: {}", i, gpu_info[i].memory_total);
  HOLOSCAN_LOG_INFO("GPU {} memory_free: {}", i, gpu_info[i].memory_free);
  HOLOSCAN_LOG_INFO("GPU {} memory_used: {}", i, gpu_info[i].memory_used);
  HOLOSCAN_LOG_INFO("GPU {} memory_usage: {}", i, gpu_info[i].memory_usage);
  HOLOSCAN_LOG_INFO("GPU {} power_limit: {}", i, gpu_info[i].power_limit);
  HOLOSCAN_LOG_INFO("GPU {} power_usage: {}", i, gpu_info[i].power_usage);
  HOLOSCAN_LOG_INFO("GPU {} temperature: {}", i, gpu_info[i].temperature);
}
```

---

## Constructors

### GPUResourceMonitor \[#gpuresourcemonitor]

```cpp showLineNumbers={false}
holoscan::GPUResourceMonitor::GPUResourceMonitor(
    uint64_t metric_flags = kDefaultGpuMetrics
)
```

Construct a new `GPUResourceMonitor` object.

This constructor creates a new `GPUResourceMonitor` object.

**Parameters**

**`metric_flags`** `uint64_t` — default: kDefaultGpuMetrics

The metric flags (default: `GPU_DEVICE_ID`)

---

### Destructor \[#destructor]

### \~GPUResourceMonitor

```cpp showLineNumbers={false}
virtual holoscan::GPUResourceMonitor::~GPUResourceMonitor()
```

---

## Methods

### init \[#init]

```cpp showLineNumbers={false}
void holoscan::GPUResourceMonitor::init()
```

Initialize the GPU resource monitor.

### close \[#close]

```cpp showLineNumbers={false}
void holoscan::GPUResourceMonitor::close()
```

Close handle of the GPU resource monitor.

This function closes the handle of the opened NVML and CUDA Runtime libraries if they are open.

### metric\_flags \[#metricflags]

#### Mutable

```cpp showLineNumbers={false}
void holoscan::GPUResourceMonitor::metric_flags(
    uint64_t metric_flags
)
```

Set metric flags.

This function sets the metric flags.

**Parameters**

**`metric_flags`** `uint64_t`

The metric flags

---

#### Const

const

```cpp showLineNumbers={false}
uint64_t holoscan::GPUResourceMonitor::metric_flags() const
```

Get metric flags.

This function returns the metric flags.

**Returns:** The metric flags.

### update \[#update]

#### Update the GPU information and cache it

```cpp showLineNumbers={false}
GPUInfo holoscan::GPUResourceMonitor::update(
    uint32_t index,
    uint64_t metric_flags = GPUMetricFlag::DEFAULT
)
```

Update the GPU information and cache it.

This function updates information for the GPU with the given index based on the given metric flags and returns the GPU information. If the metric flags are not provided, the existing metric flags are used. It also caches the GPU information.

**Returns:** The GPU information.

**Parameters**

**`index`** `uint32_t`

The GPU index.

---

**`metric_flags`** `uint64_t` — default: GPUMetricFlag::DEFAULT

The metric flags.

---

#### Update all GPU information and cache it

```cpp showLineNumbers={false}
std::vector<GPUInfo> holoscan::GPUResourceMonitor::update(
    uint64_t metric_flags = GPUMetricFlag::DEFAULT
)
```

Update all GPU information and cache it.

This function updates the information for all GPUs based on the given metric flags and returns a vector of GPU information. If the metric flags are not provided, the existing metric flags are used. It also caches the GPU information.

**Returns:** The vector of GPU information.

**Parameters**

**`metric_flags`** `uint64_t` — default: GPUMetricFlag::DEFAULT

The metric flags.

---

#### Update the GPU information

```cpp showLineNumbers={false}
GPUInfo & holoscan::GPUResourceMonitor::update(
    uint32_t index,
    GPUInfo &gpu_info,
    uint64_t metric_flags = GPUMetricFlag::DEFAULT
)
```

Update the GPU information.

This function fills the GPU information given as the argument based on the given metric flags and returns the GPU information. If the metric flags are not provided, the existing metric flags are used.

**Returns:** The GPU information filled with the updated values (same as the argument).

**Parameters**

**`index`** `uint32_t`

The GPU index.

---

**`gpu_info`** `GPUInfo &`

The GPU information.

---

**`metric_flags`** `uint64_t` — default: GPUMetricFlag::DEFAULT

The metric flags.

---

### gpu\_info \[#gpuinfo]

#### Get the GPU information

```cpp showLineNumbers={false}
GPUInfo holoscan::GPUResourceMonitor::gpu_info(
    uint32_t index,
    uint64_t metric_flags = GPUMetricFlag::DEFAULT
)
```

Get the GPU information.

This method returns the GPU information based on the given index.

If the metric flags are provided, it returns the vector of GPU information based on the given metric flags. If the metric flags are not provided, it returns the cached GPU information.

**Returns:** The GPU information.

**Parameters**

**`index`** `uint32_t`

The GPU index.

---

**`metric_flags`** `uint64_t` — default: GPUMetricFlag::DEFAULT

The metric flags.

---

#### Get all GPU information

```cpp showLineNumbers={false}
std::vector<GPUInfo> holoscan::GPUResourceMonitor::gpu_info(
    uint64_t metric_flags = GPUMetricFlag::DEFAULT
)
```

Get all GPU information.

This method returns the vector of GPU information. If the metric flags are provided, it returns the GPU information based on the given metric flags. If the metric flags are not provided, it returns the cached GPU information.

**Returns:** All GPU information.

**Parameters**

**`metric_flags`** `uint64_t` — default: GPUMetricFlag::DEFAULT

The metric flags.

---

### num\_gpus \[#numgpus]

```cpp showLineNumbers={false}
uint32_t holoscan::GPUResourceMonitor::num_gpus() const
```

Get the number of GPUs.

**Returns:** The number of GPUs.

### is\_integrated\_gpu \[#isintegratedgpu]

```cpp showLineNumbers={false}
bool holoscan::GPUResourceMonitor::is_integrated_gpu(
    uint32_t index
)
```

Check whether the GPU is integrated (iGPU).

**Returns:** True if the GPU is integrated (iGPU), false otherwise.

### bind\_nvml\_methods \[#bindnvmlmethods]

```cpp showLineNumbers={false}
bool holoscan::GPUResourceMonitor::bind_nvml_methods()
```

### bind\_cuda\_runtime\_methods \[#bindcudaruntimemethods]

```cpp showLineNumbers={false}
bool holoscan::GPUResourceMonitor::bind_cuda_runtime_methods()
```

### init\_nvml \[#initnvml]

```cpp showLineNumbers={false}
bool holoscan::GPUResourceMonitor::init_nvml()
```

### init\_cuda\_runtime \[#initcudaruntime]

```cpp showLineNumbers={false}
bool holoscan::GPUResourceMonitor::init_cuda_runtime()
```

### shutdown\_nvml \[#shutdownnvml]

```cpp showLineNumbers={false}
void holoscan::GPUResourceMonitor::shutdown_nvml() noexcept
```

### shutdown\_cuda\_runtime \[#shutdowncudaruntime]

```cpp showLineNumbers={false}
void holoscan::GPUResourceMonitor::shutdown_cuda_runtime() noexcept
```

---

## Member variables

| Name                                | Type                                        | Description                                                             |
| ----------------------------------- | ------------------------------------------- | ----------------------------------------------------------------------- |
| `handle_`                           | `void *`                                    | The handle of the GPU resource monitor.                                 |
| `cuda_handle_`                      | `void *`                                    | The handle of the CUDA Runtime library.                                 |
| `nvmlErrorString`                   | `nvml::nvmlErrorString_t`                   | The function pointer to the nvmlErrorString function.                   |
| `nvmlInit`                          | `nvml::nvmlInit_t`                          | The function pointer to the nvmlInit function.                          |
| `nvmlDeviceGetCount`                | `nvml::nvmlDeviceGetCount_t`                | The function pointer to the nvmlDeviceGetCount function.                |
| `nvmlDeviceGetHandleByIndex`        | `nvml::nvmlDeviceGetHandleByIndex_t`        | The function pointer to the nvmlDeviceGetHandleByIndex function.        |
| `nvmlDeviceGetHandleByPciBusId`     | `nvml::nvmlDeviceGetHandleByPciBusId_t`     | The function pointer to the nvmlDeviceGetHandleByPciBusId function.     |
| `nvmlDeviceGetHandleBySerial`       | `nvml::nvmlDeviceGetHandleBySerial_t`       | The function pointer to the nvmlDeviceGetHandleBySerial function.       |
| `nvmlDeviceGetHandleByUUID`         | `nvml::nvmlDeviceGetHandleByUUID_t`         | The function pointer to the nvmlDeviceGetHandleByUUID function.         |
| `nvmlDeviceGetName`                 | `nvml::nvmlDeviceGetName_t`                 | The function pointer to the nvmlDeviceGetName function.                 |
| `nvmlDeviceGetIndex`                | `nvml::nvmlDeviceGetIndex_t`                | The function pointer to the nvmlDeviceGetIndex function.                |
| `nvmlDeviceGetPciInfo`              | `nvml::nvmlDeviceGetPciInfo_t`              | The function pointer to the nvmlDeviceGetPciInfo function.              |
| `nvmlDeviceGetSerial`               | `nvml::nvmlDeviceGetSerial_t`               | The function pointer to the nvmlDeviceGetSerial function.               |
| `nvmlDeviceGetUUID`                 | `nvml::nvmlDeviceGetUUID_t`                 | The function pointer to the nvmlDeviceGetUUID function.                 |
| `nvmlDeviceGetMemoryInfo`           | `nvml::nvmlDeviceGetMemoryInfo_t`           | The function pointer to the nvmlDeviceGetMemoryInfo function.           |
| `nvmlDeviceGetUtilizationRates`     | `nvml::nvmlDeviceGetUtilizationRates_t`     | The function pointer to the nvmlDeviceGetUtilizationRates function.     |
| `nvmlDeviceGetPowerManagementLimit` | `nvml::nvmlDeviceGetPowerManagementLimit_t` | The function pointer to the nvmlDeviceGetPowerManagementLimit function. |
| `nvmlDeviceGetPowerUsage`           | `nvml::nvmlDeviceGetPowerUsage_t`           | The function pointer to the nvmlDeviceGetPowerUsage function.           |
| `nvmlDeviceGetTemperature`          | `nvml::nvmlDeviceGetTemperature_t`          | The function pointer to the nvmlDeviceGetTemperature function.          |
| `nvmlShutdown`                      | `nvml::nvmlShutdown_t`                      | The function pointer to the nvmlShutdown function.                      |
| `cudaGetErrorString`                | `cuda::cudaGetErrorString_t`                | The function pointer to the cudaGetErrorString function.                |
| `cudaGetDeviceCount`                | `cuda::cudaGetDeviceCount_t`                | The function pointer to the cudaGetDeviceCount function.                |
| `cudaGetDeviceProperties`           | `cuda::cudaGetDeviceProperties_t`           | The function pointer to the cudaGetDeviceProperties function.           |
| `cudaDeviceGetPCIBusId`             | `cuda::cudaDeviceGetPCIBusId_t`             | The function pointer to the cudaDeviceGetPCIBusId function.             |
| `cudaMemGetInfo`                    | `cuda::cudaMemGetInfo_t`                    | The function pointer to the cudaMemGetInfo function.                    |
| `metric_flags_`                     | `uint64_t`                                  | The metric flags.                                                       |
| `is_cached_`                        | `bool`                                      | The flag to indicate whether the GPU information is cached.             |
| `gpu_count_`                        | `uint32_t`                                  | The cached number of GPUs.                                              |
| `gpu_info_`                         | `std::vector< GPUInfo >`                    | The cached GPU information.                                             |
| `nvml_devices_`                     | `std::vector< nvml::nvmlDevice_t >`         | The cached NVML devices.                                                |