Is this page helpful?

NVIDIA NVSwitch#

NVIDIA NVSwitch is a high-bandwidth, low-latency fabric that connects multiple GPUs in one system. For vGPU, peer-to-peer traffic between GPUs on the fabric uses NVLink instead of relying only on PCIe. The switch behaves as a crossbar so GPUs can reach each other at NVLink rates, which matters for multi-GPU workloads that exchange data in memory on a single node.

Support is limited to specific hardware platforms, vGPU types, hypervisor releases, and guest operating systems.

For information about using the NVSwitch, refer to the NVIDIA Fabric Manager documentation.

Note

Note: On NVIDIA HGX B200 and B300 with VMware vSphere, NVSwitch support is delivered through a Linux based CRX (Container Runtime for ESXi) compatible appliance. For installation instructions, refer to Installing Fabric Manager CRX.

Platform Support for NVIDIA NVSwitch#

  • NVIDIA HGX B300 8-GPU baseboard

  • NVIDIA HGX B200 8-GPU baseboard

  • NVIDIA HGX H200 8-GPU baseboard

  • NVIDIA HGX H100 8-GPU baseboard

  • NVIDIA HGX H800 8-GPU baseboard

  • NVIDIA HGX A100 8-GPU baseboard

NVIDIA NVSwitch Limitations#

  • Only time-sliced vGPUs are supported. MIG-backed vGPUs are not supported.

  • All vGPUs communicating peer-to-peer must be assigned to the same VM.

  • On GPUs based on the NVIDIA Hopper and Blackwell GPU architectures, multicast is supported when unified memory (UVM) is enabled.

  • GPU passthrough on NVIDIA systems that include NVSwitch when using VMware vSphere is supported only as a full-node passthrough configuration. Sub-node passthrough configurations, such as 1-, 2-GPU VMs are not supported on NVIDIA HGX systems with VMware vSphere.

  • On NVIDIA HGX B200 and B300 with VMware vSphere, NVSwitch support is delivered through a Linux based CRX (Container Runtime for ESXi) compatible appliance. This appliance provides the same multi-vGPU and NVSwitch feature set as Linux KVM, including NVLink multicast with UVM enabled. Live migration is not supported when UVM is enabled.

  • On NVIDIA HGX B200 and HGX B300 with VMware vSphere, NVIDIA vGPU for Compute supports 1:1 vGPU VMs and multi-vGPU VMs only. Fractional vGPU VMs are not supported.

Hypervisor Platform Support for NVSwitch#

Consult the documentation from your hypervisor vendor for information about which generic Linux with KVM hypervisor software releases supports NVIDIA NVSwitch.

  • All supported Red Hat Enterprise Linux KVM and Ubuntu KVM releases support NVIDIA NVSwitch.

  • The earliest VMware vSphere Hypervisor (ESXi) release that supports NVIDIA NVSwitch depends on the GPU architecture.

VMware vSphere ESXi release by GPU architecture
Table 57 VMware vSphere Releases that Support NVIDIA NVSwitch#

GPU Architecture

Earliest Supported VMware vSphere Hypervisor (ESXi) Release

NVIDIA Blackwell

VMware vSphere Hypervisor (ESXi) 9 Update 1 [1]

NVIDIA Hopper

VMware vSphere Hypervisor (ESXi) 8 update 2

NVIDIA Ampere

VMware vSphere Hypervisor (ESXi) 8 update 1

vGPU Support for NVSwitch#

Only the following vGPU for Compute time-sliced vGPUs allocated all of the physical GPU’s framebuffer are supported:

  • NVIDIA A800

  • NVIDIA A100 HGX

  • NVIDIA B300 HGX

  • NVIDIA B200 HGX

  • NVIDIA H800

  • NVIDIA H200 HGX

  • NVIDIA H100 SXM5

  • NVIDIA H20

Table 58 NVIDIA NVSwitch Support on the NVIDIA Blackwell GPU Architecture#

Board

vGPU

NVIDIA HGX B300 279 GB

B300X-279C

NVIDIA B200 HGX 180 GB

B200X-180C
Table 59 NVIDIA NVSwitch Support on the NVIDIA Hopper GPU Architecture#

Board

vGPU

NVIDIA H800 SXM5 80 GB

H800XM-80C

NVIDIA H200 SXM5 141 GB

H200X-141C

NVIDIA H100 SXM5 80 GB

H100XM-80C

NVIDIA H20 SXM5 141 GB

H20X-141C

NVIDIA H20 SXM5 96 GB

H20-96C
Table 60 NVIDIA NVSwitch Support on the NVIDIA Ampere GPU Architecture#

Board

vGPU

NVIDIA A800 HGX 80 GB

A800DX-80C

NVIDIA A100 HGX 80 GB

A100DX-80C

NVIDIA A100 HGX 40 GB

A100X-40C

Guest OS Releases Support for NVSwitch#

Linux only. NVIDIA NVSwitch is not supported on Windows.