Is this page helpful?

Managing vGPUs from a hypervisor by using NVML#

NVIDIA vGPU software supports monitoring and control of physical GPUs and virtual GPUs that are running on virtualization hosts. NVML includes functions that are specific to managing vGPUs on NVIDIA vGPU software virtualization hosts. These functions are defined in the nvml_grid.h header file.

Note

NVIDIA vGPU software does not support the management of pass-through GPUs from a hypervisor. NVIDIA vGPU software supports the management of pass-through GPUs only from within the guest VM that is using them.

Determining whether a GPU supports hosting of vGPUs#

If called on platforms or GPUs that do not support NVIDIA vGPU, functions that are specific to managing vGPUs return one of the following errors:

NVML_ERROR_NOT_SUPPORTED
NVML_ERROR_INVALID_ARGUMENT

To determine whether a GPU supports hosting of vGPUs, call the nvmlDeviceGetVirtualizationMode() function.

A vGPU-capable device reports its virtualization mode as NVML_GPU_VIRTUALIZATION_MODE_HOST_VGPU.

Discovering the vGPU capabilities of a physical GPU#

To discover the vGPU capabilities of a physical GPU, call the functions in the following table.

Function	Purpose
`nvmlDeviceGetVirtualizationMode()`	Determine the virtualization mode of a GPU. GPUs capable of hosting virtual GPUs report their virtualization mode as `NVML_GPU_VIRTUALIZATION_MODE_HOST_VGPU`.
`nvmlDeviceGetSupportedVgpus()`	Return a list of vGPU type IDs that are supported by a GPU.
`nvmlDeviceGetCreatableVgpus()`	Return a list of vGPU type IDs that can currently be created on a GPU. The result reflects the number and type of vGPUs that are already running on the GPU. If no vGPUs are running on the GPU, the list contains the IDs of all vGPU types that the GPU supports. If one or more vGPUs are running on the GPU, but the GPU is not fully loaded, the list contains only the IDs of the types of the vGPUs that have the same amount of frame buffer as the vGPUs that are already running. If the GPU is fully loaded, the list contains no vGPU type IDs.
`nvmlDeviceGetActiveVgpus()`	Return a list of handles for vGPUs currently running on a GPU.
`nvmlDeviceGetVgpuMetadata()`	Return a vGPU metadata structure for the physical GPU.
`nvmlDeviceGetVgpuCapabilities()`	Determine whether the requested vGPU capability is supported by the specified physical GPU.

Getting the properties of a vGPU type#

To get the properties of a vGPU type, call the functions in the following table.

Function	Purpose
`nvmlVgpuTypeGetClass()`	Read the class of a vGPU type (for example, Quadro, or NVS)
`nvmlVgpuTypeGetName()`	Read the name of a vGPU type (for example, GRID M60-0Q)
`nvmlVgpuTypeGetDeviceID()`	Read PCI device ID of a vGPU type (vendor/device/subvendor/subsystem)
`nvmlVgpuTypeGetFramebufferSize()`	Read the frame buffer size of a vGPU type
`nvmlVgpuTypeGetNumDisplayHeads()`	Read the number of display heads supported by a vGPU type
`nvmlVgpuTypeGetResolution()`	Read the maximum resolution of a vGPU type’s supported display head
`nvmlVgpuTypeGetLicense()`	Read license information required to operate a vGPU type
`nvmlVgpuTypeGetFrameRateLimit()`	Read the static frame limit for a vGPU type
`nvmlVgpuTypeGetMaxInstances()`	Read the maximum number of vGPU instances that can be created on a GPU
`nvmlVgpuTypeGetGpuInstanceProfileId()`	Read the corresponding GPU instance profile ID of a vGPU type. Because NVIDIA vGPU software supports only GPUs that do not support MIG, this function returns `INVALID_GPU_INSTANCE_PROFILE_ID`.
`nvmlVgpuTypeGetCapabilities()`	Determine whether the vGPU type supports peer-to-peer CUDA transfers over NVLink or GPUDirect® technology.

Getting the properties of a vGPU instance#

To get the properties of a vGPU instance, call the functions in the following table.

Function	Purpose
`nvmlVgpuInstanceGetVmID()`	Read the ID of the VM currently associated with a vGPU instance
`nvmlVgpuInstanceGetUUID()`	Read a vGPU instance’s UUID
`nvmlVgpuInstanceGetMdevUUID()`	Read a vGPU instance’s virtual function I/O (VFIO) mediated device (`mdev`) UUID (hypervisors based on KVM only)
`nvmlVgpuInstanceGetVmDriverVersion()`	Read the guest driver version currently loaded on a vGPU instance
`nvmlVgpuInstanceGetFbUsage()`	Read a vGPU instance’s current frame buffer usage
`nvmlVgpuInstanceGetFBCStats()`	Read the following frame buffer capture (FBC) statistics for a vGPU instance: Count of active FBC sessions Moving average of new frames captured per second by all active sessions Moving average of new frame capture latency in microseconds for all active sessions
`nvmlVgpuInstanceGetFBCSessions()`	For each active FBC session on a vGPU instance, read the following statistics: FBC session ID Owning PID vGPU instance identifier Display ordinal associated with the FBC session. FBC session type FBC session flags Maximum horizontal resolution supported by the session Maximum vertical resolution supported by the session Horizontal resolution requested by the caller in the capture call Vertical resolution requested by the caller in the capture call Moving average of new frames captured per second by the session Moving average new frame capture latency in microseconds for the session
`nvmlVgpuInstanceGetLicenseStatus()`	Read a vGPU instance’s current license status (licensed or unlicensed)
`nvmlVgpuInstanceGetType()`	Read the vGPU type ID of a vGPU instance
`nvmlVgpuInstanceGetFrameRateLimit()`	Read a vGPU instance’s frame rate limit
`nvmlVgpuInstanceGetEncoderStats()`	Read the following encoder statistics for a vGPU instance: Count of active encoder sessions One-second trailing average of encoded FPS of all active sessions One-second trailing average of encode latency in microseconds
`nvmlVgpuInstanceGetEncoderSessions()`	For each active encoder session on a vGPU instance, read the following statistics: Encoder session ID Owning PID Codec type, for example, H.264 or H.265 Encode resolution One-second trailing averages for encoded FPS and encode latency
`nvmlDeviceGetVgpuUtilization()`	Read a vGPU instance’s usage of the following resources as a percentage of the physical GPU’s capacity: 3D/Compute Frame buffer bandwidth Video encoder Video decoder
`nvmlDeviceGetVgpuProcessUtilization()`	For each process running on a vGPU instance, read the process ID and usage by the process of the following resources as a percentage of the physical GPU’s capacity: 3D/Compute Frame buffer bandwidth Video encoder Video decoder
`nvmlDeviceGetGridLicensableFeatures()`	Return a structure containing information about whether a vGPU or a physical GPU supports NVIDIA vGPU software licensing and, if so, additional licensing information such as: The NVIDIA vGPU software products that can be licensed on the vGPU or physical GPU License status The exact name of the NVIDIA vGPU software product for which the vGPU or physical GPU is licensed for and enabled The year, month, day, hour, minutes, and seconds of the license expiration date in the GMT (UTC+0) timezone.
`nvmlVgpuInstanceGetAccountingMode()`	Read the accounting mode of the vGPU instance
`nvmlVgpuInstanceGetAccountingPids()`	Read the maximum number of processes that can be queried and the current list of process IDs
`nvmlVgpuInstanceGetAccountingStats()`	For each process ID returned by `nvmlVgpuInstanceGetAccountingPids()`, read the following statistics: GPU utilization Memory utilization Maximum memory usage Amount of time the graphics or compute context was active Start time of the process Whether the process is still running
`nvmlGetVgpuCompatibility()`	Return compatibility information about a vGPU and a physical GPU, such as: The VM states from which the vGPU can be run on the physical GPU Any factors limiting compatibility between the vGPU and the physical GPU
`nvmlVgpuInstanceGetMetadata()`	Return a vGPU metadata structure for a vGPU and its associated VM
`nvmlGetVgpuDriverCapabilities()`	Determine whether the requested vGPU driver capability is supported by the Virtual GPU manager.

Building an NVML-enabled application for a vGPU host#

Functions that are specific to vGPUs are defined in the header file nvml_grid.h.

To build an NVML-enabled application for a vGPU host, ensure that you include nvml_grid.h in addition to nvml.h:

#include <nvml.h>
#include <nvml_grid.h>

For more information, refer to the sample code that is included in the SDK.