Is this page helpful?

Optimizing VM Configuration for Performant AI Inference

This white paper provides detailed guidance on configuring virtual machines (VMs) to support AI/ML workloads when a hypervisor layer is deployed on top of HGX systems

NVIDIA AI Enterprise

• Introduction
  • About this paper
  • Problem Statement
  • Target Audience
  • Enviroment
• System Setup and Installation
  • System Configuration
  • Installing the Hypervisor
  • Understanding System Topology
    • Sample ‘lstopo’ output
    • Additional System Setup Considerations
  • Optimal Device Selection
    • Select devices on the same CPU socket / NUMA node as the VM’s Virtual CPUs
    • Select NICs, Storage Controllers proximate to the GPU
    • Select GPUs connected via NVlink
• Configuring Virtual Machines
  • VM Configuration Pre-requisites
  • Editing a VM’s XML definition
  • Scenario: Full passthrough- All GPUs, NICs, Storage Controllers, NVswitches
  • Virtual CPU configuration
    • Virtual CPU sockets and NUMA nodes
    • Virtual CPU pinning
  • Virtual PCI Express configuration
    • Virtual PCIe expander bus
    • Virtual PCIe root ports
    • Virtual PCIe switches
  • Passthrough device assignment
• Day 2 Operations
  • Deploying NVIDIA NIMs on Kubernetes
• Known Limitations
• Appendix
  • ATS and ACS configuration
    • Enabling ATS on NVIDIA NICs
    • Configuring ACS
  • Example physical topology map for HGX H200 8-GPU platform
  • Example libvirt domain XML for HGX H200 8-GPU platform
  • NCCL

Notices

• Notices
  • Notice
  • Trademarks
  • Copyright