Host Setup

Hypervisor administrator tasks: verify prerequisites, download software through the NGC CLI, install the Virtual GPU Manager, and confirm Fabric Manager on HGX servers. After these steps complete on the host, individual VM owners install the guest driver per Installing NVIDIA vGPU Guest Driver.

Prerequisites

Confirm the following before you install NVIDIA vGPU for Compute.

System Requirements

• At least one NVIDIA data center GPU in a single NVIDIA AI Enterprise compatible NVIDIA-Certified Systems. NVIDIA recommends the following GPUs based on your infrastructure.

  Table 71 System Requirements Use Cases

  Use Case	GPU
  AI Inference and Mainstream AI Servers	NVIDIA A30 NVIDIA A100 1 - 8x NVIDIA L4 NVIDIA L40S NVIDIA H100 NVL NVIDIA H200 NVL NVIDIA RTX PRO 6000 Blackwell Server Edition NVIDIA RTX PRO 4500 Blackwell Server Edition
  AI Model Training (Large) and Inference (HGX Scale Up and Out Server)	NVIDIA H100 HGX NVIDIA H200 HGX NVIDIA B200 HGX NVIDIA B300 HGX

• If using GPUs based on the NVIDIA Ampere architecture or later, the following BIOS settings are enabled on your server platform:

  • Single Root I/O Virtualization (SR-IOV) - Enabled

  • VT-d/IOMMU - Enabled

• NVIDIA AI Enterprise License

• NVIDIA AI Enterprise Software:

  • NVIDIA Virtual GPU Manager

  • NVIDIA vGPU for Compute Guest Driver

Use nvidia-smi for testing, monitoring, and benchmarking.

Recommended server settings

• Hyperthreading - Enabled

• Power Setting or System Profile - High Performance

• CPU Performance - Enterprise or High Throughput (if available in the BIOS)

• Memory Mapped I/O greater than 4-GB - Enabled (if available in the BIOS)

Installing NGC CLI

Use the NGC Catalog CLI to download NVIDIA Virtual GPU Manager and the vGPU for Compute Guest Driver from the NVIDIA NGC Catalog.

To install the NGC Catalog CLI:

1. Log in to the NVIDIA NGC Catalog.

2. In the top right corner, click Welcome and then select Setup from the menu.

3. Click Downloads under Install NGC CLI from the Setup page.

4. From the CLI Install page, click the Windows, Linux, or MacOS tab, according to the platform from which you will be running NGC Catalog CLI.

5. Follow the instructions to install the CLI.

6. Verify the installation by entering ngc --version in a terminal or command prompt. The output should be NGC Catalog CLI x.y.z where x.y.z indicates the version.

7. Configure NGC CLI so that you can run the commands. You will be prompted to enter your NGC API Key. Enter the following command:

   $ ngc config set
   
   Enter API key [no-apikey]. Choices: [<VALID_APIKEY>, 'no-apikey']: (COPY/PASTE API KEY)
   
   Enter CLI output format type [ascii]. Choices: [ascii, csv, json]: ascii
   
   Enter org [no-org]. Choices: ['no-org']:
   
   Enter team [no-team]. Choices: ['no-team']:
   
   Enter ace [no-ace]. Choices: ['no-ace']:
   
   Successfully saved NGC configuration to /home/$username/.ngc/config
   

8. In a terminal or command window, run:

   • NVIDIA Virtual GPU Manager

     ngc registry resource download-version "nvidia/vgpu/vgpu-host-driver-X:X.X"
     

   • NVIDIA vGPU for Compute Guest Driver

     ngc registry resource download-version "nvidia/vgpu/vgpu-guest-driver-X:X.X"
     

For more information on configuring the NGC CLI, refer to the Getting Started with the NGC CLI documentation.

Installing NVIDIA Virtual GPU Manager

Install Virtual GPU Manager on the hypervisor to enable GPU virtualization. Steps depend on the platform. This section assumes:

• You have downloaded the Virtual GPU Manager software from NVIDIA NGC Catalog

• You want to deploy the NVIDIA vGPU for Compute on a single server node

Table 72 Hypervisor Platform Installation Instructions for the NVIDIA Virtual GPU Manager

Hypervisor Platform	Installation Instructions
Red Hat Enterprise Linux KVM	Installing and Configuring the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM
Ubuntu KVM	Installing and Configuring the NVIDIA Virtual GPU Manager for Ubuntu
VMware vSphere	Installing and Configuring the NVIDIA Virtual GPU Manager for VMware vSphere

Next, install the vGPU Guest Driver in each guest VM per Installing NVIDIA vGPU Guest Driver.

NVIDIA Fabric Manager on HGX Servers

NVIDIA Fabric Manager coordinates NVSwitch and NVLink on NVIDIA HGX platforms for multi-GPU VMs.

Starting with NVIDIA AI Enterprise Infra 8.0, Fabric Manager and Fabric Manager development binaries are integrated into the NVIDIA AI Enterprise drivers. A separate Fabric Manager installation is no longer required. NVIDIA NVLink System Monitor (NVLSM) continues to be provided as a standalone utility.

When Fabric Manager Is Required

When Fabric Manager is required on HGX

• Required for multi-GPU VMs (1, 2, 4, or 8 GPUs) on HGX platforms

• Necessary for Ampere, Hopper, and Blackwell HGX systems with NVSwitch

• Enables high-bandwidth interconnect topologies for AI training and large-scale workloads

It provides a unified GPU memory fabric, monitors NVLinks, and supports high-bandwidth communication among GPUs in the same VM.

Note

• Fabric Manager is available after you install the NVIDIA Virtual GPU Manager or NVIDIA Data Center GPU Driver. No separate package installation is required.

• Start the Fabric Manager service before creating VMs with multi-GPU configurations. Without it on HGX, GPU topologies inside the VM may be incomplete or non-functional. For capabilities, configuration, and usage, refer to the NVIDIA Fabric Manager User Guide.

• For Fabric Manager integration or 1-, 2-, 4-, or 8-GPU VM deployment on your hypervisor, refer to your hypervisor vendor documentation.

Fabric Manager CRX for VMware ESXi (Blackwell)

On NVIDIA Blackwell platforms (B200 HGX and later), VMware ESXi hosts require the Open Fabric Daemon (OFD) for Fabric Manager to operate correctly. Because OFD is not present on current ESXi releases, Fabric Manager runs inside a CRX (Container Runtime Executive) on the ESXi host.

Installing Fabric Manager CRX

Prerequisites

• All VMs on the host must be powered off; otherwise the host cannot enter maintenance mode.

• ConnectX-7 bridges must not be in passthrough mode.

• VMware ESXi version 9.1 or later is required

Steps

1. Disable the ACS capability check so virtual functions (VFs) can be created. On VMware ESXi, VF creation commonly requires this setting. Check the current value:

   $ esxcli system settings kernel list -o disableACSCheck
   

   Disable the check:

   $ esxcli system settings kernel set -s disableACSCheck -v TRUE
   

2. Put the host in maintenance mode:

   $ esxcli system maintenanceMode set --enable true
   

3. Install the NVIDIA Fabric Manager CRX component:

   $ esxcli software component apply --no-sig-check -d /path/to/fm-crx
   

4. Exit maintenance mode:

   $ esxcli system maintenanceMode set --enable false
   

5. Verify that the Fabric Manager CRX instance is running:

   $ crx-cli list -w
         ID       CID          IMAGE      UUID    HOME
         209880   -830842625   drivervm           /var/run/crx/drivervm-20987f-hT36Zc
   

6. Check the status of the installation and the Fabric Manager service:

   $ crx-cli exec --tty <ID> /bin/cat /var/log/start_script.log
   

   Replace <ID> with the instance ID from the previous step, for example 209880. The output shows the PCI devices attached to the CRX instance and the status of the Fabric Manager service.

Verifying Device Groups

After the Fabric Manager CRX and the NVIDIA AI Enterprise host driver are installed, device groups are created automatically.

1. Check the device group configuration:

   $ cat /var/run/nvidia/deviceGroup.json
   

2. Verify successful creation in the log:

   $ cat /var/log/nvidia-deviceGroup.log
   

   On success, the log contains messages similar to the following:

   2026-02-12 08:43:39 [notice] nvidia-device-groups service started.
   2026-02-12 08:43:39 [notice] Selected hypervisor is esxi
   2026-02-12 08:43:45 [notice] Fabric Manager reported 15 partition(s)
   2026-02-12 08:43:45 [notice] Finished writing device group information to /var/run/nvidia/deviceGroup.json
   2026-02-12 08:43:45 [notice] nvidia-device-groups service terminated
   

Uninstalling Fabric Manager CRX

Prerequisites

• All VMs on the host must be powered off; otherwise the host cannot enter maintenance mode.

Steps

1. Put the host in maintenance mode:

   $ esxcli system maintenanceMode set --enable true
   

2. Remove the NVIDIA Fabric Manager CRX component:

   $ esxcli software component remove -n NVD-FM-CRX-910
   

3. Exit maintenance mode:

   $ esxcli system maintenanceMode set --enable false
   

4. Verify that the Fabric Manager CRX instance is no longer running:

   $ crx-cli list -w
      ID       CID       IMAGE      UUID    HOME
   

Troubleshooting Fabric Manager CRX

Fabric Manager process not running inside the CRX

Restart the CRX by entering and exiting maintenance mode:

$ esxcli system maintenanceMode set --enable true
$ esxcli system maintenanceMode set --enable false

Virtual functions (VFs) are not created

Confirm that the ACS capability check was disabled during install. See step 1 in Installing Fabric Manager CRX.

Note

Intermittent issues with Fabric Manager CRX may require a host reboot to resolve.

Related Topics

• Installing NVIDIA vGPU Guest Driver - guest VM driver installation (next phase)

• AI Enterprise Workload Setup - GPU Operator and AI Enterprise application containers in licensed VMs

• NVIDIA AI Enterprise Product Support Matrix - supported platforms, GPUs, and component versions

• Limitations - platform-specific known issues and host/guest constraints