NVIDIA AI Enterprise Bare Metal Deployment Guide

This document provides insights into deploying NVIDIA AI Enterprise on Bare Metal Servers and serves as a technical resource for understanding system pre-requisites, installation, and configuration.

NVIDIA AI Enterprise

• Platform Overview
  • Application Layer
    • Application Layer Software
  • Infrastructure Layer
    • Infrastructure Layer Software
    • Infra Software Branch Support Matrix
    • Infra Software Branch Release Notes

Deployment Guide

• Overview
• Prerequisites
• NVIDIA AI Enterprise Software
  • NVIDIA Driver
  • Installing NGC Catalog CLI
• NVIDIA License System
  • NVIDIA License System User Guide
• Advanced GPU configuration (Optional)
• Creating Your First NVIDIA AI Enterprise System
  • Installing an Operation System
    • Installing Ubuntu Server 20.04 LTS or 22.04 LTS
    • Installing Red Hat Enterprise Linux 8.4
  • Installing the NVIDIA Driver
    • Data Center Driver Installation
• Installing Docker and The Docker Utility Engine for NVIDIA GPUs
  • Installing Docker
  • Installing the NVIDIA Container Toolkit
  • Testing Docker and NVIDIA Container Runtime
• Installing AI and Data Science Applications and Frameworks
  • Accessing AI and Data Science Tools and Frameworks
  • NVIDIA AI Enterprise Search Filters
  • NVIDIA TensorRT
  • NVIDIA Triton Inference Server
  • NVIDIA RAPIDS
  • PyTorch
  • TensorFlow
  • TAO Toolkit
• Advance Framework Configuration
  • Securing a notebook server
  • Startup Scripts for Individual Containers
    • Startup Scripts for Jupyter
      • RAPIDS Container
      • TensorFlow1 Container
      • TensorFlow2 Container
      • PyTorch Container
    • Combined Startup Script
    • Enabling Startup Script
  • Startup Scripts for Triton Inference Server
• Validation
  • TensorRT RN50 Inference
    • Commands to the Run Test
    • Interpreting the Results

Appendix A: Kubernetes

• Overview
  • What is Kubernetes?
  • What is Helm?
  • What is the NVIDIA Network Operator?
  • What is the NVIDIA GPU Operator?
    • How does the GPU Operator help IT Infrastructure Teams?
• Disable Nouveau
  • Ubuntu
  • RHEL
• Running with Kubernetes
  • Installing Docker
  • Installing Kubernetes
    • Disable swap
  • Initializing the Kubernetes cluster to run as a control-plane node
  • Installing Helm
  • NVIDIA Network Operator
    • Prerequisites
    • Install NVIDIA Network Operator
    • Validating the State of Network Operator
  • NVIDIA GPU Operator
    • Install GPU Operator
    • License GPU Operator for CLS
    • License GPU Operator for DLS
    • GPU Operator with RDMA (Optional)
      • Prerequisites
      • Validating the Network Operator with GPUDirect RDMA
    • Validate the state of the GPU Operator

Appendix B: GPUDirect Storage

• Overview
• Prerequisites
  • Supported Systems
    • PCI Switch Systems
    • Converged Card Systems
  • BIOS and OS Settings
  • Supported Operating Systems
  • Supported File Systems
  • Supported GPUs
  • Networking
• Deployment Steps
  • Creating GDS Enabled Kubernetes
    • Install Kubernetes
    • Install Network Operator
    • Install GPU Operator with GDS support
• Creating an Application Container
  • Application Container Dockerfile
  • Kubernetes Pod YAML File
• Validate PCI Switch System Topology

Support

• Support and Services

Notices

• Notices
  • Notice
  • Trademarks
  • Copyright