NVIDIA AI Enterprise Deployment Guide

This document provides insights into deploying NVIDIA AI Enterprise for VMware vSphere and serves as a technical resource for understanding system pre-requisites, installation, and configuration.

NVIDIA AI Enterprise

• Enterprise-Grade AI Software Platform
  • Enterprise-Grade AI Software Platform From NVIDIA
  • NVIDIA AI Software Supported
  • NVIDIA AI Workflows
  • Production-Ready Pretrained Models
  • NVIDIA NIM
  • Supported Hardware and Software

Deployment Guide

• Overview
• NVIDIA AI Enterprise Software
  • NVIDIA vGPU Architecture
    • NVIDIA AI Enterprise Host Software (VIB)
    • NVIDIA vGPU Guest Driver
  • Installing NGC Catalog CLI
    • NVIDIA AI Enterprise Host Software (VIB)
    • NVIDIA vGPU Guest Driver
• Prerequisite
• Installing VMware ESXi
  • Choosing the Installation Method
  • Preparing USB Boot Media
  • Installing VMware ESXi
  • Initial Host Configuration
• Installing VMware vCenter Server
  • Installing VCenter Server Appliance (VCSA)
    • Software Considerations:
    • Architectural Considerations:
    • Hardware and Storage Requirements:
  • vCenter Server Appliance (VCSA) Installation
  • Post Installation
    • Adding Licenses to the vCenter Server
    • Adding a Host
    • Setting the NTP Service on a Host
    • Setting the CPU Power Management Policy
    • Setting a vCenter Appliance to Auto-Start
    • Mounting an NFS ISO Data Store
• Installing and Configuring NVIDIA AI Enterprise Host Software
  • Preparing the VIB file for Install
  • Uploading VIB in vSphere Web Client
  • Installing the VIB
  • Updating the VIB
  • Verifying the Installation of the VIB
  • Uninstalling the VIB
  • Changing the Default Graphics Type in VMware vSphere
• NVIDIA License System
  • NVIDIA License System User Guide
• Advanced GPU configuration (Optional)
  • GPU Partitioning
    • NVIDIA AI Enterprise Software Partitioning
      • Profiles for NVIDIA AI Enterprise
      • Scheduling Policies
      • RmPVMRL Registry Key
      • Changing the vGPU Scheduling Policy for All GPUs
      • Changing the vGPU Scheduling Policy for Select GPUs
      • Restoring Default vGPU Scheduler Settings
    • NVIDIA Multi-Instance GPU Configuration for vSphere
      • Terminology
        • GPU Context
        • GPU Engine
        • GPU Memory Slice
        • GPU SM Slice
        • GPU Slice
        • GPU Instance
        • Compute Instance
      • MIG Prerequisites
      • Enable MIG Mode
      • VM Configuration with a GPU Instance
      • VM Configuration with a GPU Instance
      • Optional: Update Containers for a MIG-Enabled vGPU
      • Destroying GPU Instances
  • GPU Aggregation for NVIDIA AI Enterprise
    • Peer-to-Peer NVIDIA NVLINK
  • Device Groups
    • Simplified Hardware Consumption with Device Groups
  • Page Retirement and ECC
  • Disabling and Enabling ECC Memory
    • Disabling ECC Memory
    • Enabling ECC Memory
• Creating Your First NVIDIA AI Enterprise VM
  • Creating a Virtual Machine
    • Installing Ubuntu Server LTS
    • Installing Red Hat Enterprise Linux
  • Enabling the NVIDIA vGPU
  • Installing the NVIDIA Driver in the Virtual Machine
    • Downloading the NVIDIA AI Enterprise Software Driver Using NGC
    • Installing the NVIDIA Driver using the .run file with Ubuntu
    • Installing the NVIDIA Driver using the .run file with RHEL
  • Licensing the VM
• Installing Docker and The Docker Utility Engine for NVIDIA GPUs
  • Installing Docker
  • Installing the NVIDIA Container Toolkit
  • Testing Docker and NVIDIA Container Runtime
• Installing AI and Data Science Applications and Frameworks
  • Accessing AI and Data Science Tools and Frameworks
  • NVIDIA AI Enterprise Search Filters
  • NVIDIA TensorRT
  • NVIDIA Triton Inference Server
  • NVIDIA RAPIDS
  • PyTorch
  • TensorFlow
  • TAO Toolkit
• Advance Framework Configuration
  • Securing a notebook server
  • Startup Scripts for Individual Containers
    • Startup Scripts for Jupyter
      • RAPIDS Container
      • TensorFlow1 Container
      • TensorFlow2 Container
      • PyTorch Container
    • Combined Startup Script
    • Enabling Startup Script
  • Startup Scripts for Triton Inference Server
• Validation
  • TensorRT RN50 Inference
    • Commands to the Run Test
    • Interpreting the Results

Appendix

• Overview
  • What is Kubernetes?
  • Why should I use vSphere with Tanzu
  • What is Helm?
  • What is the NVIDIA Network Operator?
  • What is the NVIDIA GPU Operator?
    • How does the GPU Operator help IT Infrastructure Teams?
• Disable Nouveau
  • Ubuntu
  • RHEL
• Running with Kubernetes
  • Installing Docker
  • Installing Kubernetes
    • Disable swap
  • Initializing the Kubernetes cluster to run as a control-plane node
  • Installing Helm
  • NVIDIA Network Operator
    • Prerequisites
    • Install NVIDIA Network Operator
    • Validating the State of Network Operator
  • NVIDIA GPU Operator
    • Install GPU Operator
    • License GPU Operator
    • GPU Operator with RDMA (Optional)
      • Prerequisites
      • Validating the Network Operator with GPUDirect RDMA
    • Validate the state of the GPU Operator
• Installing VMware vSphere with VMware Tanzu
  • Prerequisites
  • Provisioning a Tanzu Cluster with NVIDIA vGPU Devices
  • IT Administrator
    • Step #1: Create VM Classes
      • Create GPU Accelerated Classes
      • Create a VM Class Using MIG
      • Create a VM Class Using NVIDIA Networking
    • Step #2: Associate the VM Class with the Supervisor’s Namespace
  • DevOps Engineer
    • Step #1: Install Kubernetes CLI Tools
    • Step #2: Create a GPU Accelerated TKG Cluster
    • Step #3: Install NVIDIA Operators
      • Deploy NVIDIA Network Operator
      • Deploy NVIDIA GPU Operator
      • Validate NVIDIA GPU Operator Deployment
  • Installing VMware vSphere with VMware Tanzu (Air-gapped)
    • Prerequisites
    • Getting Started with Air-gapped Deployments
    • Deploying a Bastion Host
    • Setting Up a Local Package Repository
      • Web Server Setup
    • Setting Up a Local Container Image Registry
      • Pulling Container Images
      • Preparing the Private Registry
      • Retag the images to point to the local private registry
      • Save the images for offline upload
      • Push the newly tagged images to Harbor
      • Accessing Tanzu from the Bastion Host
    • Logging into the target Tanzu Kubernetes Cluster
      • Add the Harbor certificate to the Cluster
    • Create ConfigMaps for the GPU Operator
      • Local Package Repository Configuration
    • NVIDIA AI Enterprise License Configuration
      • Private Repository Authentication
    • Update GPU Operator Helm Chart Definitions
      • Add the ConfigMaps to the Helm Chart
      • Update the Image Specs to use the Local Registry
    • Install the GPU Operator

Support

• Support and Services

Notices

• Notices
  • Notice
  • Trademarks
  • Copyright