NVIDIA AI Enterprise Documentation

Getting Started

• Release Notes
  • What is Included in NVIDIA AI Enterprise Infra 7.2
  • Compatibility and Support
• Quick Start Guide
  • Activating the Accounts for Getting NVIDIA AI Enterprise
    • Before You Begin
    • Your Order Confirmation Message
    • NVIDIA Enterprise Account Requirements
      • Creating your NVIDIA Enterprise Account
      • Linking an Evaluation Account to an NVIDIA Enterprise Account for Purchased Licenses
  • Installing NVIDIA AI Enterprise Software Components
    • The NVIDIA NGC Catalog
      • Accessing the NVIDIA AI Enterprise Infrastructure Software
      • Accessing the NVIDIA AI Enterprise AI and Application Software
    • NVIDIA AI Enterprise Deployment Options
  • Installing NVIDIA AI Enterprise on Bare Metal Ubuntu 22.04
    • NVIDIA AI Enterprise Software Prerequisites
    • Installing NVIDIA AI Enterprise using the TRD Driver on Ubuntu 22.04 from a .run File
    • Installing the NVIDIA Container Toolkit
    • Verifying the Installation of NVIDIA Container Toolkit
    • Installing Software Distributed as Container Images
    • Running an NVIDIA Production Branch PyTorch Container
    • Running ResNet-50 with TensorRT
    • Running NVIDIA NIM on Bare Metal Ubuntu 22.04
  • Installing NVIDIA AI Enterprise on Public Cloud
    • Installing NVIDIA AI Enterprise on Microsoft Azure using the NVIDIA AI Enterprise VMI
    • Running an LLM NIM on Microsoft Azure using NVIDIA AI Enterprise
  • Installing NVIDIA AI Enterprise in Virtualized Environments using NVIDIA vGPU for Compute
    • NVIDIA AI Enterprise Software Prerequisites
  • Obtaining NVIDIA Base Command Manager
• Support Matrix
  • Prerequisites
  • Supported NVIDIA Infrastructure Software
  • Supported NVIDIA GPUs and Networking
  • Supported Platforms
  • Bare Metal
  • Virtualized
  • Base Command Manager
  • Public Cloud
    • Managed Kubernetes
    • Standard GPU Instances
    • NVIDIA GPU Optimized VMI on CSP Marketplace
  • CPU-Only Server Support
• Deployment Guide
  • NVIDIA AI Enterprise Deployment Guides
  • Use Cases and Examples

Infrastructure Software

• NVIDIA vGPU for Compute
  • Key Concepts
    • Glossary
    • NVIDIA vGPU Architecture Overview
      • NVIDIA vGPU for Compute Configurations
  • Installing NVIDIA vGPU for Compute
    • Prerequisites
      • System Requirements
      • Installing NGC CLI
    • Installing NVIDIA Virtual GPU Manager
    • Installing NVIDIA Fabric Manager on HGX Servers
    • Installing NVIDIA vGPU Guest Driver
      • Licensing a NVIDIA vGPU for Compute Guest VM
      • Verifying the License Status of a Licensed NVIDIA vGPU for Compute Guest VM
      • Installing the NVIDIA GPU Operator Using a Bash Shell Script
    • Installing NVIDIA AI Enterprise Applications Software
      • Installing NVIDIA AI Enterprise Applications Software using Docker and NVIDIA Container Toolkit
        • Prerequisites
        • Installing Docker Engine
        • Installing the NVIDIA Container Toolkit
        • Verifying the Installation: Run a Sample CUDA Container
        • Accessing NVIDIA AI Enterprise Containers on NGC
      • Installing the NVIDIA AI Enterprise Software Components Using Podman
      • Installing NVIDIA AI Enterprise Software Components Using Kubernetes and NVIDIA Cloud Native Stack
  • NVIDIA vGPU for Compute Key Features
    • MIG Backed vGPU
      • Supported MIG-Backed vGPU Configurations on a Single GPU
      • Configuring MIG-Backed vGPU
        • Configuring a GPU for MIG-Backed vGPUs
          • Enabling MIG Mode for a GPU
          • Creating GPU Instances on a MIG-Enabled GPU
          • Creating Non-Default Compute Instances in a GPU Instance for 1:1 vGPUs
          • Disabling MIG Mode for One or More GPUs
      • Modifying a MIG-Backed vGPU’s Configuration From a Guest VM
      • Monitoring MIG-backed vGPU Activity
    • Device Groups
    • GPUDirect RDMA and GPUDirect Storage
      • GPUDirect RDMA and GPUDirect Storage Known Issues and Limitations
      • Hypervisor Platform Support for GPUDirect RDMA and GPUDirect Storage
      • vGPU Support for GPUDirect RDMA and GPUDirect Storage
      • Guest OS Releases Support for GPUDirect RDMA and GPUDirect Storage
      • Network Interface Cards Support for GPUDirect RDMA and GPUDirect Storage
    • Heterogeneous vGPU
      • Platform Support for Heterogeneous vGPUs
    • Live Migration
      • Live Migration Known Issues and Limitations
      • Platform Support for Live Migration
      • vGPU Support for Live Migration
    • Multi-vGPU and P2P
      • Multi vGPU
      • Peer-To-Peer (P2P) CUDA Transfers
      • Peer-to-Peer CUDA Transfers Known Issues and Limitations
      • Hypervisor Platform Support for Multi-vGPU and P2P
      • vGPU Support for Multi-vGPU
      • vGPU Support for P2P
    • NVIDIA NVSwitch
      • Platform Support for NVIDIA NVSwitch
      • NVIDIA NVSwitch Limitations
      • Hypervisor Platform Support for NVSwitch
      • vGPU Support for NVSwitch
      • Guest OS Releases Support for NVSwitch
    • NVLink Multicast
      • vGPU Support for NVLink Multicast
    • Scheduling Policies
    • Suspend-Resume
      • Suspend-Resume Known Issues and Limitations
      • Platform Support for Suspend-Resume
      • vGPU Support for Suspend-Resume
    • Unified Virtual Memory (UVM)
      • UVM Known Issues and Limitations
      • Hypervisor Platform Support for UVM
      • vGPU Support for UVM
  • Product Limitations and Known Issues
    • Red Hat Enterprise Linux with KVM Limitations and Known Issues
    • Ubuntu KVM Limitations and Known Issues
    • VMware vSphere Limitations and Known Issues
      • Requirements for Using vGPU for Compute on VMware vSphere for GPUs Requiring 64 GB+ of MMIO Space with Large-Memory VMs
    • Microsoft Windows Server Limitations and Known Issues
  • Virtual GPU Types for Supported GPUs
    • NVIDIA Blackwell GPU Architecture
      • MIG-Backed and Time-Sliced NVIDIA vGPU for Compute for the NVIDIA Blackwell Architecture
    • NVIDIA Hopper GPU Architecture
      • MIG-Backed and Time-Sliced NVIDIA vGPU for Compute for the NVIDIA Ampere GPU Architecture
    • NVIDIA Ada Lovelace GPU Architecture
    • NVIDIA Ampere GPU Architecture
      • MIG-Backed and Time-Sliced NVIDIA vGPU for Compute for the NVIDIA Ampere GPU Architecture
    • NVIDIA Turing GPU Architecture
    • NVIDIA Volta GPU Architecture
  • vGPU for Compute FAQs