AI Enterprise Documentation

Tip

This document is also available as a PDF.

Overview

• Release Notes
  • 5.3 Release
  • 5.2 Release

Support

• Infrastructure Support Matrix
  • Supported NVIDIA Infrastructure Software
  • Supported NVIDIA GPUs and Networking
  • Supported Platforms
  • Bare Metal
  • Virtualized
  • Base Command Manager Essentials
  • Public Cloud
    • Managed Kubernetes
    • Standard GPU Instances
    • NVIDIA GPU Optimized VMI on CSP Marketplace
  • CPU-Only Server Support

Getting Started

• Quick Start Guide
  • Activating the Accounts for Getting NVIDIA AI Enterprise
    • Before You Begin
    • Your Order Confirmation Message
    • NVIDIA Enterprise Account Requirements
      • Creating your NVIDIA Enterprise Account
      • Linking an Evaluation Account to an NVIDIA Enterprise Account for Purchased Licenses
  • Installing NVIDIA AI Enterprise Software Components
    • The NVIDIA NGC Catalog
      • Accessing the NVIDIA AI Enterprise Infrastructure Resources
      • Accessing the NVIDIA AI Enterprise Software Suite
    • NVIDIA AI Enterprise Deployment Options
  • Installing NVIDIA AI Enterprise on Bare Metal Ubuntu 22.04
    • NVIDIA AI Enterprise Software Prerequisites
    • Installing NVIDIA AI Enterprise using the TRD Driver on Ubuntu 22.04 from a .run File
    • Installing the NVIDIA Container Toolkit
    • Verifying the Installation of NVIDIA Container Toolkit
    • Installing Software Distributed as Container Images
    • Running ResNet-50 with TensorFlow
    • Running ResNet-50 with TensorRT
    • Running NVIDIA NIM on Bare Metal Ubuntu 22.04
  • Installing NVIDIA AI Enterprise on Public Cloud
    • Installing NVIDIA AI Enterprise on Microsoft Azure using the NVIDIA AI Enterprise VMI
    • Running an LLM NIM on Microsoft Azure using NVIDIA AI Enterprise
  • Installing NVIDIA AI Enterprise in Virtualized Environments using NVIDIA vGPU (C-Series)
    • NVIDIA AI Enterprise Software Prerequisites
  • Obtaining NVIDIA Base Command Manager Essentials
• Deployment Guide
  • NVIDIA AI Enterprise Deployment Guides
  • Use Cases and Examples

Appendix

• NVIDIA vGPU (C-Series) Documentation
  • Release Notes
    • Supported Hardware
      • Microsoft Windows Guest Operating Systems Supported
    • Requirements for Using NVIDIA vGPU (C-Series)
    • Requirements for Using NVIDIA vGPU (C-Series) on GPUs Requiring 64 GB or More of MMIO Space with Large-Memory VMs
    • NVIDIA CUDA Toolkit Version Support
    • NVIDIA vGPU (C-Series) Migration Support
      • vGPUs that Support Multiple vGPUs Assigned to a VM
      • vGPUs that Support Peer-to-Peer CUDA Transfers
    • GPUDirect Technology Support
    • NVIDIA NVSwitch On-Chip Memory Fabric Support
      • Hardware Platforms that Support NVIDIA NVSwitch On-Chip Memory Fabric
      • vGPUs that Support NVIDIA NVSwitch On-Chip Memory Fabric
      • Hypervisor Releases that Support NVIDIA NVSwitch On-Chip Memory Fabric
      • Guest OS Releases that Support NVIDIA NVSwitch On-Chip Memory Fabric
      • Limitations on Support for NVIDIA NVSwitch On-Chip Memory Fabric
    • vGPUs that Support Unified Memory
    • nvidia-smi cannot report GPU utilization for MIG instances
    • Total frame buffer for vGPUs is less than the total frame buffer on the physical GPU
    • Single vGPU benchmark scores are lower than passthrough GPU
  • User Guide
    • MIG-Backed NVIDIA vGPU Internal Architecture
    • Valid MIG-Backed Virtual GPU Configurations on a Single GPU
    • Installing and Configuring the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM
    • Installing and Configuring the NVIDIA Virtual GPU Manager for Ubuntu
    • Installing and Configuring the NVIDIA Virtual GPU Manager for VMware vSphere
    • Configuring a GPU for MIG-Backed vGPUs
      • Enabling MIG Mode for a GPU
      • Creating GPU Instances on a MIG-Enabled GPU
      • Optional: Creating Compute Instances in a GPU instance
    • Disabling MIG Mode for One or More GPUs
    • Installing NVIDIA AI Enterprise Software Components by Using Kubernetes
      • Transforming Container Images for AI and Data Science Applications and Frameworks into Kubernetes Pods
    • Installing NVIDIA AI Enterprise Software, Applications, and Deep Learning Framework Components by Using Docker
    • Installing NVIDIA GPU Operator by Using a Bash Shell Script
    • Modifying a MIG-Backed vGPU’s Configuration
    • Monitoring MIG-backed vGPU Activity
    • Virtual GPU Types for Supported GPUs
      • NVIDIA A800 PCIe 80GB, NVIDIA A800 PCIe 80GB Liquid Cooled, and NVIDIA AX800 Virtual GPU Types
      • NVIDIA A800 PCIe 40GB Virtual GPU Types
      • NVIDIA A800 HGX Virtual GPU Types
      • NVIDIA A100 PCIe 40GB Virtual GPU Types
      • NVIDIA A100 HGX 40GB Virtual GPU Types
      • NVIDIA A100 PCIe 80GB, NVIDIA A100 PCIe 80GB Liquid-Cooled and NVIDIA A100X Virtual GPU Types
      • NVIDIA A100 HGX 80GB Virtual GPU Types
      • NVIDIA A40 Virtual GPU Types
      • NVIDIA A30, NVIDIA A30X, and NVIDIA A30 Liquid-Cooled Virtual GPU Types
      • NVIDIA A16 Virtual GPU Types
      • NVIDIA A10 Virtual GPU Types
      • NVIDIA H100 PCIe 94GB (H100 NVL) Virtual GPU Types
      • NVIDIA H100 SXM5 94GB Virtual GPU Types
      • NVIDIA H100 PCIe 80GB Virtual GPU Types
      • NVIDIA H100 SXM5 80GB Virtual GPU Types
      • NVIDIA H100 SXM5 64GB Virtual GPU Types
      • NVIDIA H800 PCIe 94GB (H800 NVL) Virtual GPU Types
      • NVIDIA H800 PCIe 80GB Virtual GPU Types
      • NVIDIA H800 SXM5 80GB Virtual GPU Types
      • NVIDIA L40 Virtual GPU Types
      • NVIDIA L40S Virtual GPU Types
      • NVIDIA L20 and NVIDIA L20 Liquid-Cooled Virtual GPU Types
      • NVIDIA L4 Virtual GPU Types
      • NVIDIA L2 Virtual GPU Types
      • NVIDIA RTX 6000 Ada Virtual GPU Types
      • NVIDIA RTX 5880 Ada Virtual GPU Types
      • NVIDIA RTX 5000 Ada Virtual GPU Types
      • NVIDIA RTX A6000 Virtual GPU Types
      • NVIDIA RTX A5500 Virtual GPU Types
      • NVIDIA RTX A5000 Virtual GPU Types
      • Tesla T4 Virtual GPU Types
      • Tesla V100 SXM2 Virtual GPU Types
      • Tesla V100 SXM2 32GB Virtual GPU Types
      • Tesla V100 PCIe Virtual GPU Types
      • Tesla V100 PCIe 32GB Virtual GPU Types
      • Tesla V100S PCIe 32GB Virtual GPU Types
      • Tesla V100 FHHL Virtual GPU Types
      • Quadro RTX 8000 Passive Virtual GPU Types
      • Quadro RTX 6000 Passive Virtual GPU Types