Is this page helpful?

Managing vGPUs from a guest VM#

NVIDIA vGPU software supports monitoring and control within a guest VM of vGPUs or pass-through GPUs that are assigned to the VM. The scope of management interfaces and tools used within a guest VM is limited to the guest VM within which they are used. They cannot monitor any other GPUs in the virtualization platform.

For monitoring from a guest VM, certain properties do not apply to vGPUs. The values that the NVIDIA vGPU software management interfaces report for these properties indicate that the properties do not apply to a vGPU.

NVIDIA vGPU Software Server Interfaces for GPU Management from a Guest VM#

The NVIDIA vGPU software server interfaces that are available for GPU management from a guest VM depend on the guest operating system that is running in the VM.

Interface	Guest OS	Notes
`nvidia-smi` command	Windows, 64-bit Linux	Command line, interactive use
NVIDIA Management Library (NVML)	Windows, 64-bit Linux	Integration of NVIDIA GPU management with third-party applications
NVIDIA Control Panel	Windows	Detailed control of graphics settings, basic configuration reporting
Windows Performance Counters	Windows	Performance metrics provided by Windows Performance Counter interfaces
NVWMI	Windows	Detailed configuration and performance metrics provided by Windows WMI interfaces

How GPU engine usage is reported#

Usage of GPU engines is reported for vGPUs as a percentage of the vGPU’s maximum possible capacity on each engine. The GPU engines are as follows:

Graphics/SM
Memory controller
Video encoder
Video decoder

The amount of a physical engine’s capacity that a vGPU is permitted to occupy depends on the scheduler under which the GPU is operating:

NVIDIA vGPUs operating under the Best Effort Scheduler and the Equal Share Scheduler are permitted to occupy the full capacity of each physical engine if no other vGPUs are contending for the same engine. Therefore, if a vGPU occupies 20% of the entire graphics engine in a particular sampling period, its graphics usage as reported inside the VM is 20%.
NVIDIA vGPUs operating under the Fixed Share Scheduler can occupy no more than their allocated share of the graphics engine. Therefore, if a vGPU has a fixed allocation of 25% of the graphics engine, and it occupies 25% of the engine in a particular sampling period, its graphics usage as reported inside the VM is 100%.

Using NVML to manage vGPUs#

NVIDIA vGPU software supports monitoring and control within a guest VM by using NVML.

Determining whether a GPU is a vGPU or pass-through GPU#

NVIDIA vGPUs are presented in guest VM management interfaces in the same fashion as pass-through GPUs.

To determine whether a GPU device in a guest VM is a vGPU or a pass-through GPU, call the NVML function nvmlDeviceGetVirtualizationMode().

A GPU reports its virtualization mode as follows:

A GPU operating in pass-through mode reports its virtualization mode as NVML_GPU_VIRTUALIZATION_MODE_PASSTHROUGH.
A vGPU reports its virtualization mode as NVML_GPU_VIRTUALIZATION_MODE_VGPU.

Physical GPU properties that do not apply to a vGPU#

Properties and metrics other than GPU engine usage are reported for a vGPU in a similar way to how the same properties and metrics are reported for a physical GPU. However, some properties do not apply to vGPUs. The NVML device query functions for getting these properties return a value that indicates that the properties do not apply to a vGPU. For details of NVML device query functions, see Device Queries in NVML API Reference Manual.

GPU identification properties that do not apply to a vGPU#

GPU Property	NVML Device Query Function	NVML return code on vGPU
Serial Number	`nvmlDeviceGetSerial()` vGPUs are not assigned serial numbers.	`NOT_SUPPORTED`
GPU UUID	`nvmlDeviceGetUUID()` vGPUs are allocated random UUIDs.	`SUCCESS`
VBIOS Version	`nvmlDeviceGetVbiosVersion()` vGPU VBIOS version is hard-wired to zero.	`SUCCESS`
GPU Part Number	`nvmlDeviceGetBoardPartNumber()`	`NOT_SUPPORTED`

`InfoROM` properties that do not apply to a vGPU#

The InfoROM object is not exposed on vGPUs. All the functions in the following table return NOT_SUPPORTED.

GPU Property	NVML Device Query Function
Image Version	`nvmlDeviceGetInforomImageVersion()`
OEM Object	`nvmlDeviceGetInforomVersion()`
ECC Object	`nvmlDeviceGetInforomVersion()`
Power Management Object	`nvmlDeviceGetInforomVersion()`

GPU operation mode properties that do not apply to a vGPU#

GPU Property	NVML Device Query Function	NVML return code on vGPU
GPU Operation Mode (Current)	`nvmlDeviceGetGpuOperationMode()` Tesla GPU operating modes are not supported on vGPUs.	`NOT_SUPPORTED`
GPU Operation Mode (Pending)	`nvmlDeviceGetGpuOperationMode()` Tesla GPU operating modes are not supported on vGPUs.	`NOT_SUPPORTED`
Compute Mode	`nvmlDeviceGetComputeMode()` A vGPU always returns `NVML_COMPUTEMODE_PROHIBITED`.	`SUCCESS`
Driver Model	`nvmlDeviceGetDriverModel()` A vGPU supports WDDM mode only in Windows VMs.	`SUCCESS` (Windows)

PCI Express properties that do not apply to a vGPU#

PCI Express characteristics are not exposed on vGPUs. All the functions in the following table return NOT_SUPPORTED.

GPU Property	NVML Device Query Function
Generation Max	`nvmlDeviceGetMaxPcieLinkGeneration()`
Generation Current	`nvmlDeviceGetCurrPcieLinkGeneration()`
Link Width Max	`nvmlDeviceGetMaxPcieLinkWidth()`
Link Width Current	`nvmlDeviceGetCurrPcieLinkWidth()`
Bridge Chip Type	`nvmlDeviceGetBridgeChipInfo()`
Bridge Chip Firmware	`nvmlDeviceGetBridgeChipInfo()`
Replays	`nvmlDeviceGetPcieReplayCounter()`
TX Throughput	`nvmlDeviceGetPcieThroughput()`
RX Throughput	`nvmlDeviceGetPcieThroughput()`

Environmental properties that do not apply to a vGPU#

All the functions in the following table return NOT_SUPPORTED.

GPU Property	NVML Device Query Function
Fan Speed	`nvmlDeviceGetFanSpeed()`
Clocks Throttle Reasons	`nvmlDeviceGetSupportedClocksThrottleReasons()` `nvmlDeviceGetCurrentClocksThrottleReasons()`
Current Temperature	`nvmlDeviceGetTemperature()` `nvmlDeviceGetTemperatureThreshold()`
Shutdown Temperature	`nvmlDeviceGetTemperature()` `nvmlDeviceGetTemperatureThreshold()`
Slowdown Temperature	`nvmlDeviceGetTemperature()` `nvmlDeviceGetTemperatureThreshold()`

Power consumption properties that do not apply to a vGPU#

vGPUs do not expose physical power consumption of the underlying GPU. All the functions in the following table return NOT_SUPPORTED.

GPU Property	NVML Device Query Function
Management Mode	`nvmlDeviceGetPowerManagementMode()`
Draw	`nvmlDeviceGetPowerUsage()`
Limit	`nvmlDeviceGetPowerManagementLimit()`
Default Limit	`nvmlDeviceGetPowerManagementDefaultLimit()`
Enforced Limit	`nvmlDeviceGetEnforcedPowerLimit()`
Min Limit	`nvmlDeviceGetPowerManagementLimitConstraints()`
Max Limit	`nvmlDeviceGetPowerManagementLimitConstraints()`

ECC properties that do not apply to a vGPU#

Error-correcting code (ECC) is not supported on vGPUs. All the functions in the following table return NOT_SUPPORTED.

GPU Property

NVML Device Query Function

Mode

nvmlDeviceGetEccMode()

Error Counts

nvmlDeviceGetMemoryErrorCounter()

nvmlDeviceGetTotalEccErrors()

Retired Pages

nvmlDeviceGetRetiredPages()

nvmlDeviceGetRetiredPagesPendingStatus()

Clocks properties that do not apply to a vGPU#

All the functions in the following table return NOT_SUPPORTED.

GPU Property	NVML Device Query Function
Application Clocks	`nvmlDeviceGetApplicationsClock()`
Default Application Clocks	`nvmlDeviceGetDefaultApplicationsClock()`
Max Clocks	`nvmlDeviceGetMaxClockInfo()`
Policy: Auto Boost	`nvmlDeviceGetAutoBoostedClocksEnabled()`
Policy: Auto Boost Default	`nvmlDeviceGetAutoBoostedClocksEnabled()`

Building an NVML-enabled application for a guest VM#

To build an NVML-enabled application, refer to the sample code included in the SDK.

Using Windows Performance Counters to monitor GPU performance#

In Windows VMs, GPU metrics are available as Windows Performance Counters through the NVIDIA GPU object.

For access to Windows Performance Counters through programming interfaces, refer to the performance counter sample code included with the NVIDIA Windows Management Instrumentation SDK.

On vGPUs, the following GPU performance counters read as 0 because they are not applicable to vGPUs:

% Bus Usage
% Cooler rate
Core Clock MHz
Fan Speed
Memory Clock MHz
PCI-E current speed to GPU Mbps
PCI-E current width to GPU
PCI-E downstream width to GPU
Power Consumption mW
Temperature C

Using NVWMI to monitor GPU performance#

In Windows VMs, Windows Management Instrumentation (WMI) exposes GPU metrics in the ROOT\CIMV2\NV namespace through NVWMI. NVWMI is included with the NVIDIA driver package. The NVWMI API Reference in Windows Help format is available for download from the NVIDIA website.

For access to NVWMI through programming interfaces, use the NVWMI SDK. The NVWMI SDK, with white papers and sample programs, is included in the NVIDIA Windows Management Instrumentation SDK.

On vGPUs, some instance properties of the following classes do not apply to vGPUs:

Gpu
PcieLink

`Gpu` instance properties that do not apply to vGPUs#

`Gpu` Instance Property	Value reported on vGPU
`gpuCoreClockCurrent`	-1
`memoryClockCurrent`	-1
`pciDownstreamWidth`	0
`pcieGpu.curGen`	0
`pcieGpu.curSpeed`	0
`pcieGpu.curWidth`	0
`pcieGpu.maxGen`	1
`pcieGpu.maxSpeed`	2500
`pcieGpu.maxWidth`	0
`power`	-1
`powerSampleCount`	-1
`powerSamplingPeriod`	-1
`verVBIOS.orderedValue`	0
`verVBIOS.strValue`	—
`verVBIOS.value`	0

`PcieLink` instance properties that do not apply to vGPUs#

No instances of PcieLink are reported for vGPU.

Enabling and disabling GPM metrics#

GPU performance monitoring (GPM) metrics are supported on all Windows guest OS releases that are supported by NVIDIA vGPU software. GPM metrics are supported only on MIG-backed vGPUs that occupy an entire GPU instance on GPUs based on the NVIDIA Blackwell architecture or a later architecture.

GPM metrics are disabled by default. To monitor GPU performance with GPM metrics, you must first enable them.

This task requires administrator privileges.

Perform this task in an elevated Windows PowerShell window (Run as administrator) in the guest VM.

Before you begin, ensure that the NVWMI service is running.

To enable GPM metrics, type the following command:
```
Invoke-CimMethod -Namespace 'root\cimv2\nv' -ClassName System -MethodName enableGpmPerfCounter
```
GPM metrics do not remain enabled after the NVWMI service or the guest VM is restarted. After the NVWMI service or the VM is restarted, enable GPM metrics again.

To disable GPM metrics, type the following command:

Invoke-CimMethod -Namespace 'root\cimv2\nv' -ClassName System -MethodName disableGpmPerfCounter

Monitoring GPU performance with GPM metrics#

GPM metrics are supported only on MIG-backed vGPUs that occupy an entire GPU instance on GPUs based on the NVIDIA Blackwell architecture or a later architecture.

Before you begin, ensure that the following prerequisites are met:

The GPU is in MIG mode.
The GPU supports GPM.
GPM metrics are enabled as explained in Enabling and disabling GPM metrics.

If any of these prerequisites is not met, any attempt to monitor GPU performance with GPM metrics returns no data.

The following table maps each GPM metric’s WMI property to its Performance Monitor counter.

WMI property	Performance Monitor counter
`graphicsUtil`	% Graphics Util
`smUtil`	% SM Util
`smOccupancy`	% SM Occupancy
`integerUtil`	% Integer Util
`anyTensorUtil`	% Any Tensor Util
`dfmaTensorUtil`	% DFMA Tensor Util
`hmmaTensorUtil`	% HMMA Tensor Util
`immaTensorUtil`	% IMMA Tensor Util
`dramBwUtil`	% DRAM BW Util
`fp64Util`	% FP64 Util
`fp32Util`	% FP32 Util
`fp16Util`	% FP16 Util
`nvdec0Util`- `nvdec7Util`	% NVDEC 0 Util-% NVDEC 7 Util
`nvjpg0Util`- `nvjpg7Util`	% NVJPG 0 Util-% NVJPG 7 Util
`nvofa0Util`- `nvofa1Util`	% NVOFA 0 Util-% NVOFA 1 Util
`nvenc0Util`- `nvenc3Util`	% NVENC 0 Util-% NVENC 3 Util

You can monitor GPU performance with GPM metrics by using either of the following methods:

Using any WMI client to query the GpmMetrics class for GPM metrics

For example, you can use the following command from a PowerShell prompt in a Windows guest VM.
```
Get-CimInstance -Namespace 'root\cimv2\nv' -ClassName GpmMetrics
```
The following example shows the query results. In the query results, instanceCount is 1. The example omits the remaining metric properties, which are listed in the preceding table.
```
id                : 1
instanceCount     : 1
gpuInstanceId     : {0}
computeInstanceId : {0}
smUtil            : {0}
smOccupancy       : {0}
...
```
Using any application that reads performance counters to read the NVIDIA GPU GPM counters

For example, you can use Windows Performance Monitor to view the counters under the NVIDIA GPU GPM object.

The object lists one instance, named in the following form:
```
GPU<index> GI<id> CI<id> (GPU Id : <id>)
```
Selecting that instance makes the GPM counters available for monitoring.

The following example shows the NVIDIA GPU GPM object in the Add Counters dialog box.

Figure 40 NVIDIA GPU GPM object in the Add Counters dialog box#

The first time you query or view GPM metrics after they are enabled, all metrics are 0 because they are derived from a pair of samples. All metrics for an idle MIG instance are also 0.