Is this page helpful?

Host Setup#

Hypervisor administrator tasks: verify prerequisites, download software through the NGC CLI, install the Virtual GPU Manager, and confirm Fabric Manager on HGX servers. After these steps complete on the host, individual VM owners install the guest driver per Installing NVIDIA vGPU Guest Driver.

Prerequisites#

Confirm the following before you install NVIDIA vGPU for Compute.

System Requirements

  • At least one NVIDIA data center GPU in a single NVIDIA AI Enterprise compatible NVIDIA-Certified Systems. NVIDIA recommends the following GPUs based on your infrastructure.

    Table 45 System Requirements Use Cases#

    Use Case

    GPU

    AI Inference and Mainstream AI Servers

    • NVIDIA A30

    • NVIDIA A100

    • 1 - 8x NVIDIA L4

    • NVIDIA L40S

    • NVIDIA H100 NVL

    • NVIDIA H200 NVL

    • NVIDIA RTX PRO 6000 Blackwell Server Edition

    • NVIDIA RTX PRO 4500 Blackwell Server Edition

    AI Model Training (Large) and Inference (HGX Scale Up and Out Server)

    • NVIDIA H100 HGX

    • NVIDIA H200 HGX

    • NVIDIA B200 HGX

    • NVIDIA B300 HGX

  • If using GPUs based on the NVIDIA Ampere architecture or later, the following BIOS settings are enabled on your server platform:

    • Single Root I/O Virtualization (SR-IOV) - Enabled

    • VT-d/IOMMU - Enabled

  • NVIDIA AI Enterprise License

  • NVIDIA AI Enterprise Software:

    • NVIDIA Virtual GPU Manager

    • NVIDIA vGPU for Compute Guest Driver

Use nvidia-smi for testing, monitoring, and benchmarking.

Recommended server settings

  • Hyperthreading - Enabled

  • Power Setting or System Profile - High Performance

  • CPU Performance - Enterprise or High Throughput (if available in the BIOS)

  • Memory Mapped I/O greater than 4-GB - Enabled (if available in the BIOS)

Installing NGC CLI#

Use the NGC Catalog CLI to download NVIDIA Virtual GPU Manager and the vGPU for Compute Guest Driver from the NVIDIA NGC Catalog.

To install the NGC Catalog CLI:

  1. Log in to the NVIDIA NGC Catalog.

  2. In the top right corner, click Welcome and then select Setup from the menu.

  3. Click Downloads under Install NGC CLI from the Setup page.

  4. From the CLI Install page, click the Windows, Linux, or MacOS tab, according to the platform from which you will be running NGC Catalog CLI.

  5. Follow the instructions to install the CLI.

  6. Verify the installation by entering ngc --version in a terminal or command prompt. The output should be NGC Catalog CLI x.y.z where x.y.z indicates the version.

  7. Configure NGC CLI so that you can run the commands. You will be prompted to enter your NGC API Key. Enter the following command:

    $ ngc config set

Enter API key [no-apikey]. Choices: [<VALID_APIKEY>, 'no-apikey']: (COPY/PASTE API KEY)

Enter CLI output format type [ascii]. Choices: [ascii, csv, json]: ascii

Enter org [no-org]. Choices: ['no-org']:

Enter team [no-team]. Choices: ['no-team']:

Enter ace [no-ace]. Choices: ['no-ace']:

Successfully saved NGC configuration to /home/$username/.ngc/config

  8. In a terminal or command window, run:

    • NVIDIA Virtual GPU Manager

      ngc registry resource download-version "nvidia/vgpu/vgpu-host-driver-X:X.X"

    • NVIDIA vGPU for Compute Guest Driver

      ngc registry resource download-version "nvidia/vgpu/vgpu-guest-driver-X:X.X"

For more information on configuring the NGC CLI, refer to the Getting Started with the NGC CLI documentation.

Installing NVIDIA Virtual GPU Manager#

Install Virtual GPU Manager on the hypervisor to enable GPU virtualization. Steps depend on the platform. This section assumes:

  • You have downloaded the Virtual GPU Manager software from NVIDIA NGC Catalog

  • You want to deploy the NVIDIA vGPU for Compute on a single server node

Table 46 Hypervisor Platform Installation Instructions for the NVIDIA Virtual GPU Manager#

Hypervisor Platform

Installation Instructions

Red Hat Enterprise Linux KVM

Installing and Configuring the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM

Ubuntu KVM

Installing and Configuring the NVIDIA Virtual GPU Manager for Ubuntu

VMware vSphere

Installing and Configuring the NVIDIA Virtual GPU Manager for VMware vSphere

Next, install the vGPU Guest Driver in each guest VM per Installing NVIDIA vGPU Guest Driver.

NVIDIA Fabric Manager on HGX Servers#

NVIDIA Fabric Manager coordinates NVSwitch and NVLink on NVIDIA HGX platforms for multi-GPU VMs.

Starting with NVIDIA AI Enterprise Infra 8.0 (vGPU 20.0), Fabric Manager and Fabric Manager development binaries are integrated into the NVIDIA AI Enterprise drivers. A separate Fabric Manager installation is no longer required. NVIDIA NVLink System Monitor (NVLSM) continues to be provided as a standalone utility.

When Fabric Manager Is Required

  • Required for multi-GPU VMs (1, 2, 4, or 8 GPUs) on HGX platforms

  • Necessary for Ampere, Hopper, and Blackwell HGX systems with NVSwitch

  • Enables high-bandwidth interconnect topologies for AI training and large-scale workloads

It provides a unified GPU memory fabric, monitors NVLinks, and supports high-bandwidth communication among GPUs in the same VM.

Note

  • Fabric Manager is available after you install the NVIDIA Virtual GPU Manager or NVIDIA Data Center GPU Driver. No separate package installation is required.

  • Start the Fabric Manager service before creating VMs with multi-GPU configurations. Without it on HGX, GPU topologies inside the VM may be incomplete or non-functional. For capabilities, configuration, and usage, refer to the NVIDIA Fabric Manager User Guide.

  • For Fabric Manager integration or 1-, 2-, 4-, or 8-GPU VM deployment on your hypervisor, refer to your hypervisor vendor documentation.