Is this page helpful?

Introduction#

About this paper#

This whitepaper provides detailed guidance for configuring a virtual machine to support AI and machine-learning workloads when KVM and libvirt are deployed on an 8-GPU NVIDIA DGX B300, which emulates an 8-GPU NVIDIA HGX B300. The hardware inventory, topology, command output, PCI addresses, CPU mapping, memory layout and domain XML are taken from the tested B300 system.

The goal is to equip datacenter administrators, cloud and platform operators, OEM and hypervisor partners, and solution architects with a reproducible topology-aware configuration. Preserving CPU, memory, GPU, NVLink and network relationships inside the guest allows communication libraries and AI frameworks to choose efficient paths. The recommendations in this whitepaper are based on a single 8-GPU B300 NVL8 system running a VM configured for full GPU and NIC passthrough. It does not publish benchmark results and it does not prescribe a model or inference container.

The physical server can also run native workloads. Bare-metal mode and VM mode use the same accelerators under exclusive ownership. Switching between the two modes is a planned maintenance operation that changes the host boot configuration and requires a reboot; see Installing the Hypervisor.. The host and guest must never manage the same GPU or NVSwitch-management function at the same time.

Scope#

This paper covers one configuration: a single NVIDIA B300 NVL8 with one KVM guest VM that owns the entire system. The guest receives all eight B300 GPUs, all sixteen ConnectX-8 physical functions and the four management ConnectX-7 functions through full passthrough, and runs Fabric Manager and NVLSM for the NVLink fabric. The paper covers:

  • Host preparation: IOMMU, boot-time VFIO binding, 1 GiB hugepages, and ATS and ACS configuration.

  • Guest CPU, NUMA and memory layout: vCPU pinning, vNUMA and hugepage backing.

  • The guest’s virtual PCIe topology and device passthrough.

  • Verification of the guest topology against the host.

  • Single-node training and inference workloads running inside the guest.

The following are not covered:

  • VMs with fewer than eight GPUs, and multiple VMs on one system, including the Shared NVSwitch virtualization model. These require NVIDIA-supported fabric partitioning; see the NVIDIA Fabric Manager User Guide.

  • vGPU, MIG and SR-IOV.

  • Passing BlueField-3 DPUs or NVMe storage controllers through to the guest, including GPUDirect Storage.

  • Multi-node configurations spanning several NVL8 systems.

Problem Statement#

AI frameworks such as PyTorch need precise information about the underlying GPU and network topology. This includes GPU placement, Network Interface Card (NIC) placement, non-uniform memory access (NUMA) relationships, PCIe switch hierarchy, and NVLink connectivity. NCCL and other communication layers use those relationships to select collective paths, while GPUDirect RDMA relies on a valid peer path between a GPU and a NIC.

A virtual machine can flatten these relationships if it exposes CPUs, memory and PCI devices without an intentional topology. The guest may then perceive remote resources as local, schedule helper threads on the wrong socket, allocate memory from the wrong NUMA node, or enumerate a GPU and NIC beneath unrelated virtual bridges. Those mistakes do not necessarily prevent the workload from starting, but they can add cross-socket traffic and lead a communication library to choose a suboptimal path, which can increase latency, and degrade the overall performance.

NVIDIA B300 NVL8 adds two details that must be represented accurately. First, eight B300 GPUs connect through two fourth-generation NVSwitches using fifth-generation NVLink. Second, the cluster network uses eight ConnectX-8 adapters, each exposing two PCIe physical functions (PFs) in the tested configuration. One integrated ConnectX-7 bridge exposes four PFs to the guest for NVSwitch management; those PFs are management endpoints, not the GPU data plane.

Target Audience#

This paper is intended for datacenter administrators, platform operators, virtualization engineers, solution architects, and partner engineers who design, deploy or maintain virtualized AI infrastructure on NVIDIA-Certified systems. It assumes Linux administration experience, familiarity with server firmware and PCIe, and working knowledge of KVM, QEMU, libvirt, hypervisor installation and device passthrough.

Environment#

Figure 1. Bare-metal and VM ownership model for the tested B300 NVL8.

Figure 1. Bare-metal and VM ownership model for the tested B300 NVL8.

Platform hardware reference: NVIDIA DGX B300 System User Guide.