Host Setup#

Hypervisor administrator tasks: verify prerequisites, download software through the NGC™ CLI, install the Virtual GPU Manager, and confirm Fabric Manager on HGX™ servers. After these steps complete on the host, individual VM owners install the guest driver per Installing NVIDIA vGPU Guest Driver.

Prerequisites#

Confirm the following before you install NVIDIA® vGPU™ for Compute.

System Requirements

  • At least one NVIDIA data center GPU in a single NVIDIA AI Enterprise compatible NVIDIA-Certified Systems. NVIDIA recommends the following GPUs based on your infrastructure.

    Table 47 System Requirements Use Cases#

    Use Case

    GPU

    AI Inference and Mainstream AI Servers

    • NVIDIA A30

    • NVIDIA A100

    • 1 - 8x NVIDIA L4

    • NVIDIA L40S

    • NVIDIA H100 NVL

    • NVIDIA H200 NVL

    • NVIDIA RTX™ PRO 6000 Blackwell Server Edition

    • NVIDIA RTX PRO 4500 Blackwell Server Edition

    AI Model Training (Large) and Inference (HGX Scale Up and Out Server)

    • NVIDIA H100 HGX

    • NVIDIA H200 HGX

    • NVIDIA B200 HGX

    • NVIDIA B300 HGX

  • If using GPUs based on the NVIDIA Ampere architecture or later, the following BIOS settings are enabled on your server platform:

    • Single Root I/O Virtualization (SR-IOV) - Enabled

    • VT-d/IOMMU - Enabled

  • NVIDIA AI Enterprise License

  • NVIDIA AI Enterprise Software:

    • NVIDIA Virtual GPU Manager

    • NVIDIA vGPU for Compute Guest Driver

Use nvidia-smi for testing, monitoring, and benchmarking.

Recommended server settings

  • Hyperthreading - Enabled

  • Power Setting or System Profile - High Performance

  • CPU Performance - Enterprise or High Throughput (if available in the BIOS)

  • Memory Mapped I/O greater than 4-GB - Enabled (if available in the BIOS)

Installing NGC CLI#

Use the NGC Catalog CLI to download NVIDIA Virtual GPU Manager and the vGPU for Compute Guest Driver from the NVIDIA NGC Catalog.

To install the NGC Catalog CLI:

  1. Log in to the NVIDIA NGC Catalog.

  2. In the top right corner, click Welcome and then select Setup from the menu.

  3. Click Downloads under Install NGC CLI from the Setup page.

  4. From the CLI Install page, click the Windows, Linux, or MacOS tab, according to the platform from which you will be running NGC Catalog CLI.

  5. Follow the instructions to install the CLI.

  6. Verify the installation by entering ngc --version in a terminal or command prompt. The output should be NGC Catalog CLI x.y.z where x.y.z indicates the version.

  7. Configure NGC CLI so that you can run the commands. The CLI prompts you to enter your NGC API Key. Enter the following command:

    $ ngc config set
    
    Enter API key [no-apikey]. Choices: [<VALID_APIKEY>, 'no-apikey']: (COPY/PASTE API KEY)
    
    Enter CLI output format type [ascii]. Choices: [ascii, csv, json]: ascii
    
    Enter org [no-org]. Choices: ['no-org']:
    
    Enter team [no-team]. Choices: ['no-team']:
    
    Enter ace [no-ace]. Choices: ['no-ace']:
    
    Successfully saved NGC configuration to /home/$username/.ngc/config
    
  8. In a terminal or command window, run:

    • NVIDIA Virtual GPU Manager

      ngc registry resource download-version "nvidia/vgpu/vgpu-host-driver-X:X.X"
      
    • NVIDIA vGPU for Compute Guest Driver

      ngc registry resource download-version "nvidia/vgpu/vgpu-guest-driver-X:X.X"
      

    Where X:X.X is the version of the driver you are downloading. For the Virtual GPU Manager and vGPU for Compute Guest Driver versions that ship with this release, refer to the component table in the 9.0 Release Notes.

For more information on configuring the NGC CLI, refer to the Getting Started with the NGC CLI documentation.

Installing NVIDIA Virtual GPU Manager#

Install Virtual GPU Manager on the hypervisor to enable GPU virtualization. Steps depend on the platform. This section assumes:

  • You have downloaded the Virtual GPU Manager software from NVIDIA NGC Catalog

  • You want to deploy the NVIDIA vGPU for Compute on a single server node

Table 48 Hypervisor Platform Installation Instructions for the NVIDIA Virtual GPU Manager#

Hypervisor Platform

Installation Instructions

Red Hat Enterprise Linux KVM

Installing and Configuring the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM

Ubuntu KVM

Installing and Configuring the NVIDIA Virtual GPU Manager for Ubuntu

VMware vSphere

Installing and Configuring the NVIDIA Virtual GPU Manager for VMware vSphere

Next, install the vGPU Guest Driver in each guest VM per Installing NVIDIA vGPU Guest Driver.

NVIDIA Fabric Manager on HGX Servers#

NVIDIA Fabric Manager coordinates NVSwitch™ and NVLink™ on NVIDIA HGX platforms for multi-GPU VMs.

Starting with NVIDIA AI Enterprise Infra 9.0, the Virtual GPU Manager KVM driver package and VMware vSphere ESXi bundle no longer ship Fabric Manager. Fabric Manager is a separate binary on all supported platforms, including Linux with KVM (Ubuntu and Red Hat Enterprise Linux) and VMware vSphere ESXi, and it is versioned on its own release line rather than the driver branch. Install the Virtual GPU Manager first, because it provides the libnvidia-fmdrv library that the Fabric Manager package depends on. On ESXi, reboot the host after you install the Fabric Manager package.

Fabric Manager remains included with the NVIDIA Data Center GPU Driver used for bare-metal and GPU passthrough deployments. NVIDIA NVLink System Monitor (NVLSM) continues to be provided as a standalone utility.

Fabric Manager Packages

Download the Fabric Manager package for your hypervisor with the other NVIDIA AI Enterprise Infra driver components for this release. In the package names, <fm_version> is the Fabric Manager release, which for Infra 9.0 is 2.0.0.rel102, and <esxi_sdk> is the ESXi SDK build that the bundle targets.

Table 49 NVIDIA Fabric Manager Packages#

Hypervisor Platform

Package

Linux with KVM

nvidia-fabricmanager-direct-<fm_version>.x86_64.rpm

Linux with KVM (development package)

nvidia-fabricmanager-direct-devel-<fm_version>.x86_64.rpm

VMware vSphere ESXi

NVD-FabricManager-x86_64-<fm_version>-1OEM.<esxi_sdk>-bundle.zip

The install commands match those for the Virtual GPU Manager on the same platform.

When Fabric Manager Is Required

When Fabric Manager is required on HGX
  • Required for multi-GPU VMs (1, 2, 4, or 8 GPUs) on HGX platforms

  • Necessary for Ampere, Hopper™, and Blackwell HGX systems with NVSwitch

  • Enables high-bandwidth interconnect topologies for AI training and large-scale workloads

It provides a unified GPU memory fabric, monitors NVLinks, and supports high-bandwidth communication among GPUs in the same VM.

Note

  • With the NVIDIA Virtual GPU Manager, install the Fabric Manager package after the Virtual GPU Manager. The install commands match those for the Virtual GPU Manager itself. With the NVIDIA Data Center GPU Driver, Fabric Manager is available after you install the driver and no separate package installation is required.

  • Start the Fabric Manager service before creating VMs with multi-GPU configurations. Without it on HGX, GPU topologies inside the VM may be incomplete or non-functional. For capabilities, configuration, and usage, refer to the NVIDIA Fabric Manager User Guide.

  • For Fabric Manager integration or 1-, 2-, 4-, or 8-GPU VM deployment on your hypervisor, refer to your hypervisor vendor documentation.

Fabric Manager CRX for VMware ESXi (Blackwell)#

On NVIDIA Blackwell platforms (B200 HGX and later), VMware ESXi hosts require the Open Fabric Daemon (OFD) for Fabric Manager to operate correctly. Because OFD is not present on current ESXi releases, Fabric Manager runs inside a CRX (Container Runtime Executive) on the ESXi host.

Note

The Fabric Manager CRX component is not included in NVIDIA AI Enterprise Infra 9.0, so the procedures in this section do not apply to this release. For NVIDIA AI Enterprise deployments on HGX B200 and HGX B300 with VMware vSphere, use NVIDIA AI Enterprise Infra 8.2. You can continue to manage the NVIDIA Virtual GPU Manager driver and daemon VIBs through the NVIDIA vCenter Plugin.

Installing Fabric Manager CRX#

Prerequisites

  • All VMs on the host must be powered off; otherwise the host cannot enter maintenance mode.

  • ConnectX®-7 bridges must not be in passthrough mode.

  • VMware ESXi version 9.1 or later is required

Steps

  1. Disable the ACS capability check so virtual functions (VFs) can be created. On VMware ESXi, VF creation commonly requires this setting. Check the current value:

    $ esxcli system settings kernel list -o disableACSCheck
    

    Disable the check:

    $ esxcli system settings kernel set -s disableACSCheck -v TRUE
    
  2. Put the host in maintenance mode:

    $ esxcli system maintenanceMode set --enable true
    
  3. Install the NVIDIA Fabric Manager CRX component:

    $ esxcli software component apply --no-sig-check -d /path/to/fm-crx
    
  4. Exit maintenance mode:

    $ esxcli system maintenanceMode set --enable false
    
  5. Verify that the Fabric Manager CRX instance is running:

    $ crx-cli list -w
          ID       CID          IMAGE      UUID    HOME
          209880   -830842625   drivervm           /var/run/crx/drivervm-20987f-hT36Zc
    
  6. Check the status of the installation and the Fabric Manager service:

    $ crx-cli exec --tty <ID> /bin/cat /var/log/start_script.log
    

    Replace <ID> with the instance ID from the previous step, for example 209880. The output shows the PCI devices attached to the CRX instance and the status of the Fabric Manager service.

Verifying Device Groups#

After the Fabric Manager CRX and the NVIDIA AI Enterprise host driver are installed, device groups are created automatically.

  1. Check the device group configuration:

    $ cat /var/run/nvidia/deviceGroup.json
    
  2. Verify successful creation in the log:

    $ cat /var/log/nvidia-deviceGroup.log
    

    On success, the log contains messages similar to the following:

    2026-02-12 08:43:39 [notice] nvidia-device-groups service started.
    2026-02-12 08:43:39 [notice] Selected hypervisor is esxi
    2026-02-12 08:43:45 [notice] Fabric Manager reported 15 partition(s)
    2026-02-12 08:43:45 [notice] Finished writing device group information to /var/run/nvidia/deviceGroup.json
    2026-02-12 08:43:45 [notice] nvidia-device-groups service terminated
    

Uninstalling Fabric Manager CRX#

Prerequisites

  • All VMs on the host must be powered off; otherwise the host cannot enter maintenance mode.

Steps

  1. Put the host in maintenance mode:

    $ esxcli system maintenanceMode set --enable true
    
  2. Remove the NVIDIA Fabric Manager CRX component:

    $ esxcli software component remove -n NVD-FM-CRX-910
    
  3. Exit maintenance mode:

    $ esxcli system maintenanceMode set --enable false
    
  4. Verify that the Fabric Manager CRX instance is no longer running:

    $ crx-cli list -w
       ID       CID       IMAGE      UUID    HOME
    

Troubleshooting Fabric Manager CRX#

Fabric Manager process not running inside the CRX

Restart the CRX by entering and exiting maintenance mode:

$ esxcli system maintenanceMode set --enable true
$ esxcli system maintenanceMode set --enable false

Virtual functions (VFs) are not created

Confirm that the ACS capability check was disabled during install. Refer to step 1 in Installing Fabric Manager CRX.

Note

Intermittent issues with Fabric Manager CRX may require a host reboot to resolve.