Prerequisite#

If you are new to virtualization, it is recommended to review VMware’s ESXi Getting Started , which includes courses and guidance on potentially any current configuration that you may already have.

  • At least one NVIDIA data center GPU in a single NVIDIA AI Enterprise Compatible NVIDIA-Certified System. NVIDIA recommends using the following based on your infrastructure.

    • Adding AI to Mainstream level servers (single to 4-GPU NVLink):

      • 1-8x L4, L40S, H100 NVL, H200 NVL

    • Large Model Inference in a Single Server (NVL2 High-Capacity AI Server):

      • 2x H200 or Blackwell GPU

    • Large Model Training and Inference (HGX Scale-Up and Out Server):

      • 4x or 8x H200, or 8x Blackwell GPU

    • If using NVIDIA A100, the following BIOS settings are required:

      • Single Root I/O Virtualization (SR-IOV) - Enabled

      • VT-d/IOMMU - Enabled

    • If GPU Infrastructure is unavailable, please refer to the CPU Deployment Guide within the Appendix.

  • NVIDIA AI Enterprise License

  • VMware ESXi Hypervisor ISO. Please refer to the latest Product Support Matrix for NVIDIA AI Enterprise.

  • Ubuntu Server 22.04 amd64 ISO. Please refer to the latest Product Support Matrix for NVIDIA AI Enterprise.

  • NVIDIA AI Enterprise Software:

    • NVIDIA AI Enterprise Host Software (VIB)

    • NVIDIA Guest Driver

Note

The NVIDIA AI Enterprise Host Software (VIB) is loaded like a driver in the vSphere hypervisor and is then managed by the vCenter Server.

You may leverage the NVIDIA System Management interface (NV-SMI) management and monitoring tool for testing and benchmarking.

The following server configuration details are considered best practices:

  • Hyperthreading - Enabled

  • Power Setting or System Profile - High Performance

  • CPU Performance (if applicable) - Enterprise or High Throughput

  • Memory Mapped I/O above 4-GB - Enabled (if applicable)

Note

If NVIDIA card detection does not include all the installed GPUs, set this option to Enabled.