Virtual GPU Software User Guide

Virtual GPU Software User Guide

Documentation for administrators that explains how to install and configure NVIDIA Virtual GPU manager, configure virtual GPU software in pass-through mode, and install drivers on guest operating systems.

NVIDIA vGPU software is a graphics virtualization platform that provides virtual machines (VMs) access to NVIDIA GPU technology.

1.1. How NVIDIA vGPU Software Is Used

NVIDIA vGPU software can be used in several ways.

1.1.1. NVIDIA vGPU

NVIDIA Virtual GPU (vGPU) enables multiple virtual machines (VMs) to have simultaneous, direct access to a single physical GPU, using the same NVIDIA graphics drivers that are deployed on non-virtualized operating systems. By doing this, NVIDIA vGPU provides VMs with unparalleled graphics performance, compute performance, and application compatibility, together with the cost-effectiveness and scalability brought about by sharing a GPU among multiple workloads.

For more information, see Installing and Configuring NVIDIA Virtual GPU Manager.

1.1.2. GPU Pass-Through

In GPU pass-through mode, an entire physical GPU is directly assigned to one VM, bypassing the NVIDA Virtual GPU Manager. In this mode of operation, the GPU is accessed exclusively by the NVIDIA driver running in the VM to which it is assigned. The GPU is not shared among VMs.

For more information, see Using GPU Pass-Through.

1.1.3. Bare-Metal Deployment

In a bare-metal deployment, you can use NVIDIA vGPU software graphics drivers with Quadro vDWS and GRID Virtual Applications licenses to deliver remote virtual desktops and applications. If you intend to use Tesla boards without a hypervisor for this purpose, use NVIDIA vGPU software graphics drivers, not other NVIDIA drivers.

To use NVIDIA vGPU software drivers for a bare-metal deployment, complete these tasks:

1. Install the driver on the physical host.

   For instructions, see Installing the NVIDIA vGPU Software Graphics Driver.

2. License any NVIDIA vGPU software that you are using.

   For instructions, see Virtual GPU Client Licensing User Guide.

3. Configure the platform for remote access.

   To use graphics features with Tesla GPUs, you must use a supported remoting solution, for example, RemoteFX, Citrix Virtual Apps and Desktops, VNC, or similar technology.

4. Use the display settings feature of the host OS to configure the Tesla GPU as the primary display.

   NVIDIA Tesla generally operates as a secondary device on bare-metal platforms.

5. If the system has multiple display adapters, disable display devices connected through adapters that are not from NVIDIA.

   You can use the display settings feature of the host OS or the remoting solution for this purpose. On NVIDIA GPUs, including Tesla GPUs, a default display device is enabled.

   Users can launch applications that require NVIDIA GPU technology for enhanced user experience only after displays that are driven by NVIDIA adapters are enabled.

1.2. How this Guide Is Organized

Virtual GPU Software User Guide is organized as follows:

• This chapter introduces the architecture and features of NVIDIA vGPU software.
• Installing and Configuring NVIDIA Virtual GPU Manager provides a step-by-step guide to installing and configuring vGPU on supported hypervisors.
• Using GPU Pass-Through explains how to configure a GPU for pass-through on supported hypervisors.
• Installing the NVIDIA vGPU Software Graphics Driver explains how to install NVIDIA vGPU software graphics driver on Windows and Linux operating systems.
• Licensing an NVIDIA vGPU explains how to license NVIDIA vGPU licensed products on Windows and Linux operating systems.
• Modifying a VM's NVIDIA vGPU Configuration explains how to remove a VM’s vGPU configuration and modify GPU assignments for vGPU-enabled VMs.
• Monitoring GPU Performance covers performance monitoring of physical GPUs and virtual GPUs from the hypervisor and from within individual guest VMs.
• Citrix Hypervisor vGPU Management covers vGPU management on Citrix Hypervisor.
• Citrix Hypervisor Performance Tuning covers vGPU performance optimization on Citrix Hypervisor.
• Troubleshooting provides guidance on troubleshooting.

1.3. NVIDIA vGPU Architecture

The high-level architecture of NVIDIA vGPU is illustrated in Figure 1. Under the control of the NVIDIA Virtual GPU Manager running under the hypervisor, NVIDIA physical GPUs are capable of supporting multiple virtual GPU devices (vGPUs) that can be assigned directly to guest VMs.

Guest VMs use NVIDIA vGPUs in the same manner as a physical GPU that has been passed through by the hypervisor: an NVIDIA driver loaded in the guest VM provides direct access to the GPU for performance-critical fast paths, and a paravirtualized interface to the NVIDIA Virtual GPU Manager is used for non-performant management operations.

Figure 1. NVIDIA vGPU System Architecture

Each NVIDIA vGPU is analogous to a conventional GPU, having a fixed amount of GPU framebuffer, and one or more virtual display outputs or “heads”. The vGPU’s framebuffer is allocated out of the physical GPU’s framebuffer at the time the vGPU is created, and the vGPU retains exclusive use of that framebuffer until it is destroyed.

All vGPUs resident on a physical GPU share access to the GPU’s engines including the graphics (3D), video decode, and video encode engines.

Figure 2. NVIDIA vGPU Internal Architecture

1.4. Supported GPUs

NVIDIA vGPU is available as a licensed product on supported Tesla GPUs. For a list of recommended server platforms and supported GPUs, consult the release notes for supported hypervisors at NVIDIA Virtual GPU Software Documentation.

1.4.1. Virtual GPU Types

The number of physical GPUs that a board has depends on the board. Each physical GPU can support several different types of virtual GPU (vGPU). vGPU types have a fixed amount of frame buffer, number of supported display heads, and maximum resolutions¹. They are grouped into different series according to the different classes of workload for which they are optimized. Each series is identified by the last letter of the vGPU type name.

The number after the board type in the vGPU type name denotes the amount of frame buffer that is allocated to a vGPU of that type. For example, a vGPU of type M60-2Q is allocated 2048 Mbytes of frame buffer on a Tesla M60 board.

Due to their differing resource requirements, the maximum number of vGPUs that can be created simultaneously on a physical GPU varies according to the vGPU type. For example, a Tesla M60 board can support up to 4 M60-2Q vGPUs on each of its two physical GPUs, for a total of 8 vGPUs, but only 2 M60-4Q vGPUs, for a total of 4 vGPUs.

When enabled, the frame-rate limiter (FRL) limits the maximum frame rate in frames per second (FPS) for a vGPU as follows:

• For B-series vGPUs, the maximum frame rate is 45 FPS.
• For Q-series, C-series, and A-series vGPUs, the maximum frame rate is 60 FPS.

By default, the FRL is enabled for all GPUs. The FRL is disabled when the vGPU scheduling behavior is changed from the default best-effort scheduler on GPUs that support alternative vGPU schedulers. For details, see Changing vGPU Scheduling Behavior. On vGPUs that use the best-effort scheduler, the FRL can be disabled as explained in the release notes for your chosen hypervisor at NVIDIA Virtual GPU Software Documentation.

1.4.1.1. Tesla M60 Virtual GPU Types

Physical GPUs per board: 2

Q-Series Virtual GPU Types for Tesla M60

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla M60

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

A-Series Virtual GPU Types for Tesla M60

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.2. Tesla M10 Virtual GPU Types

Physical GPUs per board: 4

Q-Series Virtual GPU Types for Tesla M10

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla M10

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

A-Series Virtual GPU Types for Tesla M10

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.3. Tesla M6 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla M6

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla M6

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

A-Series Virtual GPU Types for Tesla M6

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.4. Tesla P100 PCIe 12GB Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla P100 PCIe 12GB

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla P100 PCIe 12GB

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla P100 PCIe 12GB

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla P100 PCIe 12GB

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.5. Tesla P100 PCIe 16GB Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla P100 PCIe 16GB

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla P100 PCIe 16GB

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla P100 PCIe 16GB

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla P100 PCIe 16GB

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.6. Tesla P100 SXM2 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla P100 SXM2

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla P100 SXM2

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla P100 SXM2

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla P100 SXM2

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.7. Tesla P40 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla P40

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla P40

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla P40

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla P40

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.8. Tesla P6 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla P6

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla P6

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla P6

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla P6

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.9. Tesla P4 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla P4

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla P4

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla P4

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla P4

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.10. Tesla T4 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla T4

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla T4

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla T4

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla T4

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.11. Tesla V100 SXM2 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla V100 SXM2

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla V100 SXM2

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla V100 SXM2

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla V100 SXM2

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.12. Tesla V100 SXM2 32GB Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla V100 SXM2 32GB

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla V100 SXM2 32GB

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla V100 SXM2 32GB

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla V100 SXM2 32GB

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.13. Tesla V100 PCIe Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla V100 PCIe

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla V100 PCIe

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla V100 PCIe

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla V100 PCIe

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.14. Tesla V100 PCIe 32GB Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla V100 PCIe 32GB

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla V100 PCIe 32GB

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla V100 PCIe 32GB

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla V100 PCIe 32GB

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.15. Tesla V100S PCIe 32GB Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla V100S PCIe 32GB

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla V100S PCIe 32GB

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla V100S PCIe 32GB

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla V100S PCIe 32GB

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.16. Tesla V100 FHHL Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Tesla V100 FHHL

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Tesla V100 FHHL

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Tesla V100 FHHL

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Tesla V100 FHHL

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.17. Quadro RTX 8000 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Quadro RTX 8000

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Quadro RTX 8000

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Quadro RTX 8000

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Quadro RTX 8000

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.18. Quadro RTX 8000 Passive Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Quadro RTX 8000 Passive

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Quadro RTX 8000 Passive

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Quadro RTX 8000 Passive

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Quadro RTX 8000 Passive

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.19. Quadro RTX 6000 Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Quadro RTX 6000

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Quadro RTX 6000

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Quadro RTX 6000

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Quadro RTX 6000

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.1.20. Quadro RTX 6000 Passive Virtual GPU Types

Physical GPUs per board: 1

Q-Series Virtual GPU Types for Quadro RTX 6000 Passive

Required license edition: Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

B-Series Virtual GPU Types for Quadro RTX 6000 Passive

Required license edition: GRID Virtual PC or Quadro vDWS

These vGPU types support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPU types. The maximum number of displays per vGPU is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see Virtual Display Resolutions for Q-series and B-series vGPUs.

For Windows 7 guest VMs, the maximum supported resolution is limited to 4096×2160 per display, regardless of whether the vGPU can support higher resolutions.

C-Series Virtual GPU Types for Quadro RTX 6000 Passive

Required license edition: vCS or Quadro vDWS

These vGPU types support a single display with a fixed maximum resolution.

A-Series Virtual GPU Types for Quadro RTX 6000 Passive

Required license edition: GRID Virtual Application

These vGPU types support a single display with a fixed maximum resolution.

1.4.2. Virtual Display Resolutions for Q-series and B-series vGPUs

Instead of a fixed maximum resolution per display, Q-series and B-series vGPUs support a maximum combined resolution based on the number of available pixels, which is determined by their frame buffer size. You can choose between using a small number of high resolution displays or a larger number of lower resolution displays with these vGPUs.

The number of virtual displays that you can use depends on a combination of the following factors:

• Virtual GPU series
• GPU architecture
• vGPU frame buffer size
• Display resolution

Various factors affect the consumption of the GPU frame buffer, which can impact the user experience. These factors include and are not limited to the number of displays, display resolution, workload and applications deployed, remoting solution, and guest OS. The ability of a vGPU to drive a certain combination of displays does not guarantee that enough frame buffer remains free for all applications to run. If applications run out of frame buffer, consider changing your setup in one of the following ways:

• Switching to a vGPU type with more frame buffer
• Using fewer displays
• Using lower resolution displays

The maximum number of displays per vGPU listed in Virtual GPU Types is based on a configuration in which all displays have the same resolution. For examples of configurations with a mixture of display resolutions, see the subsections that follow.

1.4.2.1. Mixed Display Configurations for B-Series vGPUs

1.4.2.2. Mixed Display Configurations for Q-Series vGPUs Based on the NVIDIA Maxwell Architecture

1.4.2.3. Mixed Display Configurations for Q-Series vGPUs Based on Architectures after NVIDIA Maxwell

1.4.3. Homogeneous Virtual GPUs

This release of NVIDIA vGPU supports only homogeneous virtual GPUs. At any given time, the virtual GPUs resident on a single physical GPU must be all of the same type. However, this restriction doesn’t extend across physical GPUs on the same card. Different physical GPUs on the same card may host different types of virtual GPU at the same time, provided that the vGPU types on any one physical GPU are the same.

For example, a Tesla M60 card has two physical GPUs, and can support several types of virtual GPU. Figure 3 shows the following examples of valid and invalid virtual GPU configurations on Tesla M60:

• A valid configuration with M60-2Q vGPUs on GPU 0 and M60-4Q vGPUs on GPU 1
• A valid configuration with M60-1B vGPUs on GPU 0 and M60-2Q vGPUs on GPU 1
• An invalid configuration with mixed vGPU types on GPU 0

Figure 3. Example vGPU Configurations on Tesla M60

1.5. Guest VM Support

NVIDIA vGPU supports Windows and Linux guest VM operating systems. The supported vGPU types depend on the guest VM OS.

For details of the supported releases of Windows and Linux, and for further information on supported configurations, see the driver release notes for your hypervisor at NVIDIA Virtual GPU Software Documentation.

1.5.1. Windows Guest VM Support

Windows guest VMs are supported only on Q-series, B-series, and A-series NVIDIA vGPU types. They are not supported on C-series NVIDIA vGPU types.

1.5.2. Linux Guest VM support

64-bit Linux guest VMs are supported only on Q-series, C-series, and B-series NVIDIA vGPU types. They are not supported on A-series NVIDIA vGPU types.

1.6. NVIDIA vGPU Software Features

NVIDIA vGPU software includes Quadro vDWS, vCS, GRID Virtual PC, and GRID Virtual Applications.

1.6.1. API Support on NVIDIA vGPU

NVIDIA vGPU includes support for the following APIs:

• Open Computing Language (OpenCL™ software) 1.2
• OpenGL^® 4.6
• Vulkan^® 1.1
• DirectX 11
• DirectX 12 (Windows 10)
• Direct2D
• DirectX Video Acceleration (DXVA)
• NVIDIA^® CUDA^® 10.2
• NVIDIA vGPU software SDK (remote graphics acceleration)
• NVIDIA RTX (on GPUs based on the NVIDIA Volta graphic architecture and later architectures)

1.6.2. NVIDIA CUDA Toolkit and OpenCL Support on NVIDIA vGPU Software

OpenCL and CUDA applications are supported on the following NVIDIA vGPU types:

• The 8Q vGPU type on Tesla M6, Tesla M10, and Tesla M60 GPUs
• All Q-series vGPU types on the following GPUs:
  • Tesla P4
  • Tesla P6
  • Tesla P40
  • Tesla P100 SXM2 16 GB
  • Tesla P100 PCIe 16 GB
  • Tesla P100 PCIe 12 GB
  • Tesla V100 SXM2
  • Tesla V100 SXM2 32GB
  • Tesla V100 PCIe
  • Tesla V100 PCIe 32GB
  • Tesla V100S PCIe 32GB
  • Tesla V100 FHHL
  • Tesla T4
  • Quadro RTX 6000
  • Quadro RTX 6000 passive
  • Quadro RTX 8000
  • Quadro RTX 8000 passive
• All C-series vGPU types

NVIDIA vGPU supports the following NVIDIA CUDA Toolkit features if the vGPU type, physical GPU, and the hypervisor software version support the feature:

• Error-correcting code (ECC) memory
• Peer-to-peer CUDA transfers over NVLink
  Note:

  To determine the NVLink topology between physical GPUs in a host or vGPUs assigned to a VM, run the following command from the host or VM:

  Copy

  Copied!

              
  
              
  $ nvidia-smi topo -m
          

Dynamic page retirement is supported for all vGPU types on physical GPUs that support ECC memory, even if ECC memory is disabled on physical GPU.

NVIDIA vGPU does not support the following NVIDIA CUDA Toolkit features:

• Unified Memory
• GPUDirect^® technology remote direct memory access (RDMA)
• Development tools such as IDEs, debuggers, profilers, and utilities as listed under CUDA Toolkit Major Components in CUDA Toolkit 10.2 Release Notes for Windows, Linux, and Mac OS
• Tracing and profiling through the CUDA Profiling Tools Interface (CUPTI)

For more information about NVIDIA CUDA Toolkit, see CUDA Toolkit 10.2 Documentation.

1.6.3. Additional Quadro vDWS Features

In addition to the features of GRID Virtual PC and GRID Virtual Applications, Quadro vDWS provides the following features:

• Workstation-specific graphics features and accelerations
• Certified drivers for professional applications
• GPU pass through for workstation or professional 3D graphics

  In pass-through mode, Quadro vDWS supports multiple virtual display heads at resolutions up to 8K and flexible virtual display resolutions based on the number of available pixels. For details, see Display Resolutions for Physical GPUs.

• 10-bit color for Windows users. (HDR/10-bit color is not currently supported on Linux, NvFBC capture is supported but deprecated.)

1.6.4. NVIDIA GPU Cloud (NGC) Containers Support on NVIDIA vGPU Software

NVIDIA vGPU software supports NGC containers in NVIDIA vGPU and GPU pass-through deployments on all supported hypervisors.

In NVIDIA vGPU deployments, the following vGPU types are supported only on GPUs based on NVIDIA GPU architectures after the Maxwell architecture:

• All Q-series vGPU types
• All C-series vGPU types

In GPU pass-through deployments, all GPUs based on NVIDIA GPU architectures after the NVIDIA Maxwell™ architecture that support NVIDIA vGPU software are supported.

The Ubuntu guest operating system is supported.

For more information about setting up NVIDIA vGPU software for use with NGC containers, see Using NGC with NVIDIA Virtual GPU Software Setup Guide.

The process for installing and configuring NVIDIA Virtual GPU Manager depends on the hypervisor that you are using. After you complete this process, you can install the display drivers for your guest OS and license any NVIDIA vGPU software licensed products that you are using.

2.1. Prerequisites for Using NVIDIA vGPU

Before proceeding, ensure that these prerequisites are met:

• You have a server platform that is capable of hosting your chosen hypervisor and NVIDIA GPUs that support NVIDIA vGPU software.
• One or more NVIDIA GPUs that support NVIDIA vGPU software is installed in your server platform.
• You have downloaded the NVIDIA vGPU software package for your chosen hypervisor, which consists of the following software:
  • NVIDIA Virtual GPU Manager for your hypervisor
  • NVIDIA vGPU software graphics drivers for supported guest operating systems
• The following software is installed according to the instructions in the software vendor's documentation:
  • Your chosen hypervisor, for example, Citrix Hypervisor, Red Hat Enterprise Linux KVM, Red Hat Virtualization (RHV), or VMware vSphere Hypervisor (ESXi)
  • The software for managing your chosen hypervisor, for example, Citrix XenCenter management GUI, or VMware vCenter Server
  • The virtual desktop software that you will use with virtual machines (VMs) running NVIDIA Virtual GPU, for example, Citrix Virtual Apps and Desktops, or VMware Horizon
  Note:

  If you are using VMware vSphere Hypervisor (ESXi), ensure that the ESXi host on which you will configure a VM with NVIDIA vGPU is not a member of a fully automated VMware Distributed Resource Scheduler (DRS) cluster. For more information, see Installing and Configuring the NVIDIA Virtual GPU Manager for VMware vSphere.

• A VM to be enabled with vGPU is created.
  Note:

  All hypervisors covered in this guide support multiple vGPUs in a VM.

• Your chosen guest OS is installed in the VM.

For information about supported hardware and software, and any known issues for this release of NVIDIA vGPU software, refer to the Release Notes for your chosen hypervisor:

• Virtual GPU Software for Citrix Hypervisor Release Notes
• Virtual GPU Software for Red Hat Enterprise Linux with KVM Release Notes
• Virtual GPU Software for VMware vSphere Release Notes

2.2. Switching the Mode of a Tesla M60 or M6 GPU

Tesla M60 and M6 GPUs support compute mode and graphics mode. NVIDIA vGPU requires GPUs that support both modes to operate in graphics mode.

Recent Tesla M60 GPUs and M6 GPUs are supplied in graphics mode. However, your GPU might be in compute mode if it is an older Tesla M60 GPU or M6 GPU, or if its mode has previously been changed.

If your GPU supports both modes but is in compute mode, you must use the gpumodeswitch tool to change the mode of the GPU to graphics mode. If you are unsure which mode your GPU is in, use the gpumodeswitch tool to find out the mode.

For more information, see gpumodeswitch User Guide.

2.3. Installing and Configuring the NVIDIA Virtual GPU Manager for Citrix Hypervisor

The following topics step you through the process of setting up a single Citrix Hypervisor VM to use NVIDIA vGPU. After the process is complete, you can install the graphics driver for your guest OS and license any NVIDIA vGPU software licensed products that you are using.

These setup steps assume familiarity with the Citrix Hypervisor skills covered in Citrix Hypervisor Basics.

2.3.1. Installing and Updating the NVIDIA Virtual GPU Manager for Citrix Hypervisor

The NVIDIA Virtual GPU Manager runs in the Citrix Hypervisor dom0 domain. The NVIDIA Virtual GPU Manager for Citrix Hypervisor is supplied as an RPM file and as a Supplemental Pack.

2.3.1.1. Installing the RPM package for Citrix Hypervisor

The RPM file must be copied to the Citrix Hypervisor dom0 domain prior to installation (see Copying files to dom0).

1. Use the rpm command to install the package:
   Copy

   Copied!

               
   
               
   [root@xenserver ~]# rpm -iv NVIDIA-vGPU-xenserver-7.0-440.121.x86_64.rpm
   Preparing packages for installation...
   NVIDIA-vGPU-xenserver-7.0-440.121
   [root@xenserver ~]#
           

2. Reboot the Citrix Hypervisor platform:
   Copy

   Copied!

               
   
               
   [root@xenserver ~]# shutdown –r now
   
   Broadcast message from root (pts/1) (Fri Oct 16 14:24:11 2020):
   
   The system is going down for reboot NOW! 
   [root@xenserver ~]#
           

2.3.1.2. Updating the RPM Package for Citrix Hypervisor

If an existing NVIDIA Virtual GPU Manager is already installed on the system and you want to upgrade, follow these steps:

1. Shut down any VMs that are using NVIDIA vGPU.
2. Install the new package using the –U option to the rpm command, to upgrade from the previously installed package:
   Copy

   Copied!

               
   
               
   [root@xenserver ~]# rpm -Uv NVIDIA-vGPU-xenserver-7.0-440.121.x86_64.rpm
   Preparing packages for installation...
   NVIDIA-vGPU-xenserver-7.0-440.121
   [root@xenserver ~]#
           

   Note:

   You can query the version of the current NVIDIA Virtual GPU Manager package using the rpm –q command:

   Copy

   Copied!

               
   
               
   [root@xenserver ~]# rpm –q NVIDIA-vGPU-xenserver-7.0-440.121
   [root@xenserver ~]#
   If an existing NVIDIA GRID package is already installed and you don’t select the upgrade (-U) option when installing a newer GRID package, the rpm command will return many conflict errors.
   Preparing packages for installation...
           file /usr/bin/nvidia-smi from install of NVIDIA-vGPU-xenserver-7.0-440.121.x86_64 conflicts with file from package NVIDIA-vGPU-xenserver-7.0-440.107.x86_64
           file /usr/lib/libnvidia-ml.so from install of NVIDIA-vGPU-xenserver-7.0-440.121.x86_64 conflicts with file from package NVIDIA-vGPU-xenserver-7.0-440.107.x86_64
           ...
           

3. Reboot the Citrix Hypervisor platform:
   Copy

   Copied!

               
   
               
   [root@xenserver ~]# shutdown –r now
   Broadcast message from root (pts/1) (Fri Oct 16 14:24:11 2020):
   
   The system is going down for reboot NOW! 
   [root@xenserver ~]#
           

2.3.1.3. Installing or Updating the Supplemental Pack for Citrix Hypervisor

XenCenter can be used to install or update Supplemental Packs on Citrix Hypervisor hosts. The NVIDIA Virtual GPU Manager supplemental pack is provided as an ISO.

1. Select Install Update from the Tools menu.
2. Click Next after going through the instructions on the Before You Start section.
3. Click Select update or supplemental pack from disk on the Select Update section and open NVIDIA’s Citrix Hypervisor Supplemental Pack ISO.

   Figure 4. NVIDIA vGPU Manager supplemental pack selected in XenCenter

   

4. Click Next on the Select Update section.
5. In the Select Servers section select all the Citrix Hypervisor hosts on which the Supplemental Pack should be installed on and click Next.
6. Click Next on the Upload section once the Supplemental Pack has been uploaded to all the Citrix Hypervisor hosts.
7. Click Next on the Prechecks section.
8. Click Install Update on the Update Mode section.
9. Click Finish on the Install Update section.

Figure 5. Successful installation of NVIDIA vGPU Manager supplemental pack

2.3.1.4. Verifying the Installation of the NVIDIA vGPU Software for Citrix Hypervisor Package

After the Citrix Hypervisor platform has rebooted, verify the installation of the NVIDIA vGPU software package for Citrix Hypervisor.

1. Verify that the NVIDIA vGPU software package is installed and loaded correctly by checking for the NVIDIA kernel driver in the list of kernel loaded modules.
   Copy

   Copied!

               
   
               
   [root@xenserver ~]# lsmod | grep nvidia
   nvidia               9522927  0
   i2c_core               20294  2 nvidia,i2c_i801
   [root@xenserver ~]#
           

2. Verify that the NVIDIA kernel driver can successfully communicate with the NVIDIA physical GPUs in your system by running the nvidia-smi command. The nvidia-smi command is described in more detail in NVIDIA System Management Interface nvidia-smi.

Running the nvidia-smi command should produce a listing of the GPUs in your platform.

Copy

Copied!

            

            
[root@xenserver ~]# nvidia-smi
Fri Oct 16 18:46:50 2020
+------------------------------------------------------+
| NVIDIA-SMI 440.121    Driver Version: 440.118.02     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla M60           On   | 00000000:05:00.0 Off |                  Off |
| N/A   25C    P8    24W / 150W |     13MiB /  8191MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla M60           On   | 00000000:06:00.0 Off |                  Off |
| N/A   24C    P8    24W / 150W |     13MiB /  8191MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla M60           On   | 00000000:86:00.0 Off |                  Off |
| N/A   25C    P8    25W / 150W |     13MiB /  8191MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla M60           On   | 00000000:87:00.0 Off |                  Off |
| N/A   28C    P8    24W / 150W |     13MiB /  8191MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

[root@xenserver ~]#
        

If nvidia-smi fails to run or doesn’t produce the expected output for all the NVIDIA GPUs in your system, see Troubleshooting for troubleshooting steps.

2.3.2. Configuring a Citrix Hypervisor VM with Virtual GPU

To support applications and workloads that are compute or graphics intensive, you can add multiple vGPUs to a single VM.

For details about which Citrix Hypervisor versions and NVIDIA vGPUs support the assignment of multiple vGPUs to a VM, see Virtual GPU Software for Citrix Hypervisor Release Notes.

Citrix Hypervisor supports configuration and management of virtual GPUs using XenCenter, or the xe command line tool that is run in a Citrix Hypervisor dom0 shell. Basic configuration using XenCenter is described in the following sections. Command line management using xe is described in Citrix Hypervisor vGPU Management.

1. Ensure the VM is powered off.
2. Right-click the VM in XenCenter, select Properties to open the VM’s properties, and select the GPU property. The available GPU types are listed in the GPU type drop-down list:

   Figure 6. Using Citrix XenCenter to configure a VM with a vGPU

   

After you have configured a Citrix Hypervisor VM with a vGPU, start the VM, either from XenCenter or by using xe vm-start in a dom0 shell. You can view the VM’s console in XenCenter.

After the VM has booted, install the NVIDIA vGPU software graphics driver as explained in Installing the NVIDIA vGPU Software Graphics Driver.

2.4. Installing the Virtual GPU Manager Package for Linux KVM

NVIDIA vGPU software for Linux Kernel-based Virtual Machine (KVM) (Linux KVM) is intended only for use with supported versions of Linux KVM hypervisors. For details about which Linux KVM hypervisor versions are supported, see Virtual GPU Software for Generic Linux with KVM Release Notes.

Before installing the Virtual GPU Manager package for Linux KVM, ensure that the following prerequisites are met:

• The following packages are installed on the Linux KVM server:

  • The x86_64 build of the GNU Compiler Collection (GCC)
  • Linux kernel headers

• The package file is copied to a directory in the file system of the Linux KVM server.

If the Nouveau driver for NVIDIA graphics cards is present, disable it before installing the package.

1. Change to the directory on the Linux KVM server that contains the package file.
   Copy

   Copied!

               
   
               
   # cd package-file-directory
           

   package-file-directory

   The path to the directory that contains the package file.

2. Make the package file executable.
   Copy

   Copied!

               
   
               
   # chmod +x package-file-name
           

   package-file-name

   The name of the file that contains the Virtual GPU Manager package for Linux KVM, for example NVIDIA-Linux-x86_64-390.42-vgpu-kvm.run.

3. Run the package file as the root user.
   Copy

   Copied!

               
   
               
   # sudo sh./package-file-name
           

   The package file should launch and display the license agreement.
4. Accept the license agreement to continue with the installation.
5. When installation has completed, select OK to exit the installer.
6. Reboot the Linux KVM server.
   Copy

   Copied!

               
   
               
   # systemctl reboot
           

2.5. Installing and Configuring the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM or RHV

The following topics step you through the process of setting up a single Red Hat Enterprise Linux Kernel-based Virtual Machine (KVM) or Red Hat Virtualization (RHV) VM to use NVIDIA vGPU.

Red Hat Enterprise Linux KVM and RHV use the same Virtual GPU Manager package, but are configured with NVIDIA vGPU in different ways.

For RHV, follow this sequence of instructions:

1. Installing the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM or RHV
2. Adding a vGPU to a Red Hat Virtualization (RHV) VM

For Red Hat Enterprise Linux KVM, follow this sequence of instructions:

1. Installing the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM or RHV
2. Getting the BDF and Domain of a GPU on Red Hat Enterprise Linux KVM
3. Creating an NVIDIA vGPU on Red Hat Enterprise Linux KVM
4. Adding One or More vGPUs to a Red Hat Enterprise Linux KVM VM
5. Setting vGPU Plugin Parameters on Red Hat Enterprise Linux KVM

After the process is complete, you can install the graphics driver for your guest OS and license any NVIDIA vGPU software licensed products that you are using.

2.5.1. Installing the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM or RHV

The NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM and Red Hat Virtualization (RHV) is provided as a .rpm file.

2.5.1.1. Installing the Virtual GPU Manager Package for Red Hat Enterprise Linux KVM or RHV

Before installing the RPM package for Red Hat Enterprise Linux KVM or RHV, ensure that the sshd service on the Red Hat Enterprise Linux KVM or RHV server is configured to permit root login. If the Nouveau driver for NVIDIA graphics cards is present, disable it before installing the package. For instructions, see How to disable the Nouveau driver and install the Nvidia driver in RHEL 7 (Red Hat subscription required).

Some versions of Red Hat Enterprise Linux KVM have z-stream updates that break Kernel Application Binary Interface (kABI) compatibility with the previous kernel or the GA kernel. For these versions of Red Hat Enterprise Linux KVM, the following Virtual GPU Manager RPM packages are supplied:

• A package for the GA Linux KVM kernel
• A package for the updated z-stream kernel

To differentiate these packages, the name of each RPM package includes the kernel version. Ensure that you install the RPM package that is compatible with your Linux KVM kernel version.

1. Securely copy the RPM file from the system where you downloaded the file to the Red Hat Enterprise Linux KVM or RHV server.
   • From a Windows system, use a secure copy client such as WinSCP.
   • From a Linux system, use the scp command.
2. Use secure shell (SSH) to log in as root to the Red Hat Enterprise Linux KVM or RHV server.
   Copy

   Copied!

               
   
               
   # ssh root@kvm-server
           

   kvm-server

   The host name or IP address of the Red Hat Enterprise Linux KVM or RHV server.

3. Change to the directory on the Red Hat Enterprise Linux KVM or RHV server to which you copied the RPM file.
   Copy

   Copied!

               
   
               
   # cd rpm-file-directory
           

   rpm-file-directory

   The path to the directory to which you copied the RPM file.

4. Use the rpm command to install the package.
   Copy

   Copied!

               
   
               
   # rpm -iv NVIDIA-vGPU-rhel-7.5-440.121.x86_64.rpm
   Preparing packages for installation...
   NVIDIA-vGPU-rhel-7.5-440.121
   #
           

5. Reboot the Red Hat Enterprise Linux KVM or RHV server.
   Copy

   Copied!

               
   
               
   # systemctl reboot
           

2.5.1.2. Verifying the Installation of the NVIDIA vGPU Software for Red Hat Enterprise Linux KVM or RHV

After the Red Hat Enterprise Linux KVM or RHV server has rebooted, verify the installation of the NVIDIA vGPU software package for Red Hat Enterprise Linux KVM or RHV.

1. Verify that the NVIDIA vGPU software package is installed and loaded correctly by checking for the VFIO drivers in the list of kernel loaded modules.
   Copy

   Copied!

               
   
               
   # lsmod | grep vfio
   nvidia_vgpu_vfio       27099  0
   nvidia              12316924  1 nvidia_vgpu_vfio
   vfio_mdev              12841  0
   mdev                   20414  2 vfio_mdev,nvidia_vgpu_vfio
   vfio_iommu_type1       22342  0
   vfio                   32331  3 vfio_mdev,nvidia_vgpu_vfio,vfio_iommu_type1
   #
           

2. Verify that the libvirtd service is active and running.
   Copy

   Copied!

               
   
               
   # service libvirtd status
           

3. Verify that the NVIDIA kernel driver can successfully communicate with the NVIDIA physical GPUs in your system by running the nvidia-smi command. The nvidia-smi command is described in more detail in NVIDIA System Management Interface nvidia-smi.

Running the nvidia-smi command should produce a listing of the GPUs in your platform.

Copy

Copied!

            

            
# nvidia-smi
Fri Oct 16 18:46:50 2020
+------------------------------------------------------+
| NVIDIA-SMI 440.121    Driver Version: 440.118.02     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla M60           On   | 0000:85:00.0     Off |                  Off |
| N/A   23C    P8    23W / 150W |     13MiB /  8191MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla M60           On   | 0000:86:00.0     Off |                  Off |
| N/A   29C    P8    23W / 150W |     13MiB /  8191MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla P40           On   | 0000:87:00.0     Off |                  Off |
| N/A   21C    P8    18W / 250W |     53MiB / 24575MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
#
        

If nvidia-smi fails to run or doesn’t produce the expected output for all the NVIDIA GPUs in your system, see Troubleshooting for troubleshooting steps.

2.5.2. Adding a vGPU to a Red Hat Virtualization (RHV) VM

Ensure that the VM to which you want to add the vGPU is shut down.

1. Determine the mediated device type (mdev_type) identifiers of the vGPU types available on the RHV host.
   Copy

   Copied!

               
   
               
   # vdsm-client Host hostdevListByCaps
   ...
     "mdev": {
                  "nvidia-155": {
                      "name": "GRID M10-2B",
                      "available_instances": "4"
                  },
                  "nvidia-36": {
                      "name": "GRID M10-0Q",
                      "available_instances": "16"
                  },
     ...
           

   The preceding example shows the mdev_type identifiers of the following vGPU types:
   • For the GRID M10-2B vGPU type, the mdev_type identifier is nvidia-155.
   • For the GRID M10-0Q vGPU type, the mdev_type identifier is nvidia-36.
2. Note the mdev_type identifier of the vGPU type that you want to add.
3. Log in to the RHV Administration Portal.
4. From the Main Navigation Menu, choose Compute > Virtual Machines > virtual-machine-name.

   virtual-machine-name

   The name of the virtual machine to which you want to add the vGPU.

5. Click Edit.
6. In the Edit Virtual Machine window that opens, click Show Advanced Options and in the list of options, select Custom Properties.
7. From the drop-down list, select mdev_type.
8. In the text field, type the mdev_type identifier of the vGPU type that you want to add and click OK.

   

After adding a vGPU to an RHV VM, start the VM.

After the VM has booted, install the NVIDIA vGPU software graphics driver as explained in Installing the NVIDIA vGPU Software Graphics Driver.

2.5.3. Getting the BDF and Domain of a GPU on Red Hat Enterprise Linux KVM

Sometimes when configuring a physical GPU for use with NVIDIA vGPU software, you must find out which directory in the sysfs file system represents the GPU. This directory is identified by the domain, bus, slot, and function of the GPU.

For more information about the directory in the sysfs file system represents a physical GPU, see NVIDIA vGPU Information in the sysfs File System.

1. Obtain the PCI device bus/device/function (BDF) of the physical GPU.
   Copy

   Copied!

               
   
               
   # lspci | grep NVIDIA
           

   The NVIDIA GPUs listed in this example have the PCI device BDFs 06:00.0 and 07:00.0.

   Copy

   Copied!

               
   
               
   # lspci | grep NVIDIA
   06:00.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M10] (rev a1)
   07:00.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M10] (rev a1)
           

2. Obtain the full identifier of the GPU from its PCI device BDF.
   Copy

   Copied!

               
   
               
   # virsh nodedev-list --cap pci| grep transformed-bdf
           

   transformed-bdf

   The PCI device BDF of the GPU with the colon and the period replaced with underscores, for example, 06_00_0.

   This example obtains the full identifier of the GPU with the PCI device BDF 06:00.0.

   Copy

   Copied!

               
   
               
   # virsh nodedev-list --cap pci| grep 06_00_0
   pci_0000_06_00_0
           

3. Obtain the domain, bus, slot, and function of the GPU from the full identifier of the GPU.
   Copy

   Copied!

               
   
               
   virsh nodedev-dumpxml full-identifier| egrep 'domain|bus|slot|function'
           

   full-identifier

   The full identifier of the GPU that you obtained in the previous step, for example, pci_0000_06_00_0.

   This example obtains the domain, bus, slot, and function of the GPU with the PCI device BDF 06:00.0.

   Copy

   Copied!

               
   
               
   # virsh nodedev-dumpxml pci_0000_06_00_0| egrep 'domain|bus|slot|function'
       <domain>0x0000</domain>
       <bus>0x06</bus>
       <slot>0x00</slot>
       <function>0x0</function>
         <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
           

2.5.4. Creating an NVIDIA vGPU on Red Hat Enterprise Linux KVM

For each vGPU that you want to create, perform this task in a Linux command shell on the Red Hat Enterprise Linux KVM host.

Before you begin, ensure that you have the domain, bus, slot, and function of the GPU on which you are creating the vGPU. For instructions, see Getting the BDF and Domain of a GPU on Red Hat Enterprise Linux KVM.

1. Change to the mdev_supported_types directory for the physical GPU.
   Copy

   Copied!

               
   
               
   # cd /sys/class/mdev_bus/domain\:bus\:slot.function/mdev_supported_types/
           

   domain

   bus

   slot

   function

   The domain, bus, slot, and function of the GPU, without the 0x prefix.

   This example changes to the mdev_supported_types directory for the GPU with the domain 0000 and PCI device BDF 06:00.0.

   Copy

   Copied!

               
   
               
   # cd /sys/bus/pci/devices/0000\:06\:00.0/mdev_supported_types/
           

2. Find out which subdirectory of mdev_supported_types contains registration information for the vGPU type that you want to create.
   Copy

   Copied!

               
   
               
   # grep -l "vgpu-type" nvidia-*/name
           

   vgpu-type

   The vGPU type, for example, M10-2Q.

   This example shows that the registration information for the M10-2Q vGPU type is contained in the nvidia-41 subdirectory of mdev_supported_types.

   Copy

   Copied!

               
   
               
   # grep -l "M10-2Q" nvidia-*/name
   nvidia-41/name
           

3. Confirm that you can create an instance of the vGPU type on the physical GPU.
   Copy

   Copied!

               
   
               
   # cat subdirectory/available_instances
           

   subdirectory

   The subdirectory that you found in the previous step, for example, nvidia-41.

   The number of available instances must be at least 1. If the number is 0, either an instance of another vGPU type already exists on the physical GPU, or the maximum number of allowed instances has already been created.

   This example shows that four more instances of the M10-2Q vGPU type can be created on the physical GPU.

   Copy

   Copied!

               
   
               
   # cat nvidia-41/available_instances
   4
           

4. Generate a correctly formatted universally unique identifier (UUID) for the vGPU.
   Copy

   Copied!

               
   
               
   # uuidgen
   aa618089-8b16-4d01-a136-25a0f3c73123
           

5. Write the UUID that you obtained in the previous step to the create file in the registration information directory for the vGPU type that you want to create.
   Copy

   Copied!

               
   
               
   # echo "uuid"> subdirectory/create
           

   uuid

   The UUID that you generated in the previous step, which will become the UUID of the vGPU that you want to create.

   subdirectory

   The registration information directory for the vGPU type that you want to create, for example, nvidia-41.

   This example creates an instance of the M10-2Q vGPU type with the UUID aa618089-8b16-4d01-a136-25a0f3c73123.

   Copy

   Copied!

               
   
               
   # echo "aa618089-8b16-4d01-a136-25a0f3c73123" > nvidia-41/create
           

   An mdev device file for the vGPU is added is added to the parent physical device directory of the vGPU. The vGPU is identified by its UUID.

   The /sys/bus/mdev/devices/ directory contains a symbolic link to the mdev device file.

6. Confirm that the vGPU was created.
   Copy

   Copied!

               
   
               
   # ls -l /sys/bus/mdev/devices/
   total 0
   lrwxrwxrwx. 1 root root 0 Nov 24 13:33 aa618089-8b16-4d01-a136-25a0f3c73123 -> ../../../devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:09.0/0000:06:00.0/aa618089-8b16-4d01-a136-25a0f3c73123
           

2.5.5. Adding One or More vGPUs to a Red Hat Enterprise Linux KVM VM

To support applications and workloads that are compute or graphics intensive, you can add multiple vGPUs to a single VM.

For details about which Red Hat Enterprise Linux KVM versions and NVIDIA vGPUs support the assignment of multiple vGPUs to a VM, see Virtual GPU Software for Red Hat Enterprise Linux with KVM Release Notes.
Ensure that the following prerequisites are met:

• The VM to which you want to add the vGPUs is shut down.
• The vGPUs that you want to add have been created as explained in Creating an NVIDIA vGPU on Red Hat Enterprise Linux KVM.

You can add vGPUs to a Red Hat Enterprise Linux KVM VM by using any of the following tools:

• The virsh command
• The QEMU command line

After adding vGPUs to a Red Hat Enterprise Linux KVM VM, start the VM.

Copy

Copied!

            

            
# virsh start vm-name
        

vm-name

The name of the VM that you added the vGPUs to.

After the VM has booted, install the NVIDIA vGPU software graphics driver as explained in Installing the NVIDIA vGPU Software Graphics Driver.

2.5.5.1. Adding One or More vGPUs to a Red Hat Enterprise Linux KVM VM by Using virsh

1. In virsh, open for editing the XML file of the VM that you want to add the vGPU to.
   Copy

   Copied!

               
   
               
   # virsh edit vm-name
           

   vm-name

   The name of the VM to that you want to add the vGPUs to.

2. For each vGPU that you want to add to the VM, add a device entry in the form of an address element inside the source element to add the vGPU to the guest VM.
   Copy

   Copied!

               
   
               
   <device>
   ...
     <hostdev mode='subsystem' type='mdev' model='vfio-pci'>
       <source>
         <address uuid='uuid'/>
       </source>
     </hostdev>
   </device>
           

   uuid

   The UUID that was assigned to the vGPU when the vGPU was created.

   This example adds a device entry for the vGPU with the UUID a618089-8b16-4d01-a136-25a0f3c73123.

   Copy

   Copied!

               
   
               
   <device>
   ...
     <hostdev mode='subsystem' type='mdev' model='vfio-pci'>
       <source>
         <address uuid='a618089-8b16-4d01-a136-25a0f3c73123'/>
       </source>
     </hostdev>
   </device>
           

   This example adds device entries for two vGPUs with the following UUIDs: