NVIDIA AI Enterprise Bare Metal Deployment Guide

This document provides insights into deploying NVIDIA AI Enterprise with Red Hat OpenShift on bare metal servers. This technical resource can be used for understanding system pre-requisites, installation, and configuration.

NVIDIA AI Enterprise

• Enterprise-Grade AI Software Platform
  • Enterprise-Grade AI Software Platform From NVIDIA
  • NVIDIA AI Software Supported
  • NVIDIA AI Workflows
  • Production-Ready Pretrained Models
  • NVIDIA NIM
  • Supported Hardware and Software

Deployment Guide

• Overview
• Prerequisites
• NVIDIA AI Enterprise Software
  • NVIDIA Driver
  • Installing NGC Catalog CLI
• NVIDIA License System
  • NVIDIA License System User Guide
• Advanced GPU configuration (Optional)
• Creating Your First NVIDIA AI Enterprise System
  • Installing an Operation System
    • Installing Ubuntu Server 20.04 LTS or 22.04 LTS
    • Installing Red Hat Enterprise Linux 8.4
  • Installing the NVIDIA Driver
    • Install Steps for CLS Scenario
      • Installing the NVIDIA Driver
        • Downloading the NVIDIA AI Enterprise Software Driver Using NGC
        • Installing the NVIDIA Driver using the .run file with Ubuntu
        • Installing the NVIDIA Driver using the .run file with RHEL
        • Licensing the NVIDIA Driver
    • Install Steps for DLS Scenario
      • Installing the NVIDIA Driver
        • Downloading the NVIDIA AI Enterprise Software Driver
        • Installing the NVIDIA Driver using the .run file with Ubuntu
        • Installing the NVIDIA Driver using the .run file with RHEL
• Installing Docker and The Docker Utility Engine for NVIDIA GPUs
  • Installing Docker
  • Installing the NVIDIA Container Toolkit
  • Testing Docker and NVIDIA Container Runtime
• Installing AI and Data Science Applications and Frameworks
  • Accessing AI and Data Science Tools and Frameworks
  • NVIDIA AI Enterprise Search Filters
  • NVIDIA TensorRT
  • NVIDIA Triton Inference Server
  • NVIDIA RAPIDS
  • PyTorch
  • TensorFlow
  • TAO Toolkit
• Advance Framework Configuration
  • Securing a notebook server
  • Startup Scripts for Individual Containers
    • Startup Scripts for Jupyter
      • RAPIDS Container
      • TensorFlow1 Container
      • TensorFlow2 Container
      • PyTorch Container
    • Combined Startup Script
    • Enabling Startup Script
  • Startup Scripts for Triton Inference Server
• Validation
  • TensorRT RN50 Inference
    • Commands to the Run Test
    • Interpreting the Results

Appendix A: Kubernetes

• Overview
  • What is Kubernetes?
  • What is Helm?
  • What is the NVIDIA Network Operator?
  • What is the NVIDIA GPU Operator?
    • How does the GPU Operator help IT Infrastructure Teams?
• Disable Nouveau
  • Ubuntu
  • RHEL
• Running with Kubernetes
  • Installing Docker
  • Installing Kubernetes
    • Disable swap
  • Initializing the Kubernetes cluster to run as a control-plane node
  • Installing Helm
  • NVIDIA Network Operator
    • Prerequisites
    • Install NVIDIA Network Operator
    • Validating the State of Network Operator
  • NVIDIA GPU Operator
    • Install GPU Operator
    • License GPU Operator for CLS
    • License GPU Operator for DLS
    • GPU Operator with RDMA (Optional)
      • Prerequisites
      • Validating the Network Operator with GPUDirect RDMA
    • Validate the state of the GPU Operator

Appendix B: GPUDirect Storage

• Overview
• Prerequisites
  • Supported Systems
    • PCI Switch Systems
    • Converged Card Systems
  • BIOS and OS Settings
  • Supported Operating Systems
  • Supported File Systems
  • Supported GPUs
  • Networking
• Deployment Steps
  • Creating GDS Enabled Kubernetes
    • Install Kubernetes
    • Install Network Operator
    • Install GPU Operator with GDS support
• Creating an Application Container
  • Application Container Dockerfile
  • Kubernetes Pod YAML File
• Validate PCI Switch System Topology

Support

• Support and Services

Notices

• Notices
  • Notice
  • Trademarks
  • Copyright