Device Groups#
Device Groups provide an abstraction layer for multi-device virtual hardware provisioning. They enable platforms to automatically detect sets of physically connected devices (such as GPUs linked through NVLink or GPU-NIC pairs) at the hardware level and present them as a single logical unit to VMs. This abstraction ensures that AI workloads that depend on low-latency, high-bandwidth communication, such as distributed model training, inference, and large-scale data processing, maximize utilization of the underlying hardware topology.
Device groups consist of two or more hardware devices that share a common PCIe switch or a direct interconnect. This simplifies virtual hardware assignment and enables:
Capability |
Description |
|---|---|
Optimized Multi-GPU and GPU-NIC communication |
NVLink-connected GPUs can be provisioned together to maximize peer-to-peer bandwidth and minimize latency, ideal for large-batch training and NCCL all-reduce-heavy workloads. Similarly, GPU-NIC pairs located under the same PCIe switch - or pairs capable of delivering optimal GPUDirect RDMA performance - are grouped together. This grouping enables high-throughput data ingestion directly into GPU memory for training or inference workloads. Adjacent NICs that do not meet the required performance thresholds are automatically excluded to avoid bottlenecks. |
Topology consistency |
Unlike manual device assignment, Device Groups guarantee correct placement across PCIe switches and interconnects, even after reboots or events like live migration. |
Simplified and reliable provisioning |
By abstracting the PCIe and NVLink topology into logical units, device groups eliminate the need for scripting or topology mapping, reducing the risk of misconfiguration and enabling faster deployment of AI clusters. |
This figure shows how devices (GPUs and NICs) that share a common PCIe switch or a direct GPU interconnect can be presented as a device group. On the right side, although two NICs are connected to the same PCIe switch as the GPU, only one NIC is included in the device group. This is because the NVIDIA driver identifies and exposes only the GPU-NIC pairings that meet the necessary criteria like GPUDirect RDMA. Adjacent NICs that do not satisfy these requirements are excluded.
For more information regarding Hypervisor Platform support for Device Groups, refer to the vGPU Device Groups documentation.