DGX OS 5 / DGX Software Stack for Ubuntu 20.04

• Introduction
  
  • DGX OS 5 Features
  • Overview
  • Additional Documentation
  • NVIDIA Enterprise Support
• Release Information
  
  • Release Cadence
  • Release Versions
  • Release Mechanisms
    
    • The ISO Image
    • The Linux Software Repositories
• Release Notes
  
  • Current Software Versions
  • DGX OS Releases
    
    • DGX OS 5.4
    • DGX OS 5.3
    • DGX OS 5.2
    • DGX OS 5.1
    • DGX OS 5.0
  • Update History
    
    • Update: November 22, 2022
    • Update: October 14, 2022
    • DGX OS 5.4 Release: August 8, 2023
    • Update: June 7, 2022
    • Update: May 17, 2022
    • DGX OS 5.3 Release: April 28, 2022
    • DGX OS 5.2 Release: February 17, 2022
    • Update: December 14, 2021
    • Update: October 26 , 2021
    • DGX OS 5.1 Release: August 26, 2021
    • Update: June 30 , 2021
    • Update: June 20 , 2021
    • Update: June 2, 2021
    • Update: May 27, 2021
    • Update: May 06, 2021
    • Update: April 20, 2021
    • Update: April 13, 2021
    • Update: March 30, 2021
    • Update: March 2, 2021
    • Update: February 23, 2021
    • Update: January 20, 2021
    • Update: December 11, 2020
    • Update: October 31, 2020 (DGX OS 5.0 Release)
  • DGX OS ISO Releases
    
    • DGX OS ISO 5.4.1
    • DGX OS ISO 5.3.1
    • DGX OS ISO 5.2.0
    • DGX OS ISO 5.1.0
    • DGX OS ISO 5.0.2
    • DGX OS ISO 5.0.0
• Installing DGX OS
  
  • Obtaining the DGX OS ISO Image
  • Installing the DGX OS Image
    
    • Installing the DGX OS Image Remotely through the BMC
    • Installing the DGX OS Image from a USB Flash Drive or DVD-ROM
      
      • Creating a Bootable USB Flash Drive by Using the dd Command
      • Creating a Bootable USB Flash Drive by Using Akeo Rufus
    • Booting the DGX OS ISO image
  • DGX OS ISO Boot Options
    
    • Install DGX OS and Reformat the Data RAID
    • Install DGX OS without Reformatting the Data RAID
    • Advanced Installation Options (Encrypted Root)
    • Boot Into a Live Environment
    • Check Disc for Defects
• Initial DGX OS Setup
  
  • Connecting to the DGX System
  • First Boot Setup Wizard
    
    • First Boot Process for DGX Servers
    • First Boot Process for DGX Station
  • Post-Installation Tasks
    
    • Adding Support for Additional Languages to the DGX Station
    • Configuring your DGX Station
    • Configuring your DGX Station V100
    • Enabling Multiple Users to Remotely Access the DGX System
• Installing the DGX Software Stack
• Upgrading DGX OS
  
  • DGX OS 5 Release Upgrade Advisements
  • Getting Release Information for DGX Systems
  • Preparing to Upgrade the Software
    
    • Connect to the DGX System Console
    • Verifying the DGX System Connection to the Repositories
  • Rotating the GPG Keys
    
    • Rotating the GPG Key For a Default Installation or After Reimaging
    • Rotating the GPG Keys for the DGX Software Stack
  • Performing a Release Upgrade from DGX OS 4
    
    • Upgrade DGX OS 4 to the Latest Version
    • Performing the Release Upgrade
    • Resolving Release Upgrade Conflicts
    • Verifying the Upgrade
    • Recovering from an Interrupted or Failed Update
  • Performing Package Upgrades
    
    • Enabling Extended Security Maintenance Upgrades
    • Performing Package Upgrades Using the CLI
    • Managing Software Upgrades on DGX Station
    • Performing Package Upgrades Using the GUI
    • Checking for Updates to DGX Station Software
• System Configuration
  
  • Network Configuration
    
    • Configuration Network Proxies
    • For the OS and Most Applications
    • For the apt Package Manager
  • Docker Configuration
    
    • Preparing the DGX System to be Used With Docker
    • Enabling Users To Run Docker Containers
    • Configuring Docker IP Addresses
    • Connectivity Requirements for NGC Containers
    • Configuring Static IP Addresses for the Network Ports
  • Managing CPU Mitigations
    
    • Determining the CPU Mitigation State of the DGX System
    • Disabling CPU Mitigations
    • Re-enable CPU Mitigations
  • Managing the DGX Crash Dump Feature
    
    • Using the Script
  • Connecting to Serial Over LAN
  • Filesystem Quotas
  • Running Workloads on Systems with Mixed Types of GPUs
    
    • Running with Docker Containers
    • Running on Bare Metal
  • Using Multi-Instance GPUs
  • Data Storage Configuration
    
    • Using Data Storage for NFS Caching
    • Using cachefilesd
    • Disabling cachefilesd
    • Changing the RAID Configuration for Data Drives
  • Running NGC Containers
    
    • Obtaining an NGC Account
    • Running NGC Containers with GPU Support
• Upgrading or Installing Additional Software
  
  • Upgrading the System
    
    • Changing Your GPU Branch
  • Checking the Currently Installed Driver Branch
  • Determining the New Available Driver Branches
  • Upgrading Your GPU Branch
    
    • Installing or Upgrading to a Newer CUDA Toolkit Release
  • CUDA Compatibility Matrix and Forward Compatibility
  • Checking the Currently Installed CUDA Toolkit Release
  • Installing or Upgrading the CUDA Toolkit
    
    • Installing or Upgrading GPUDirect Storage
  • Configuring IOMMU
  • Installing GPUDirect Storage
  • Installing nvidia_peermem
  • Installing nvidia-gds
• Known Issues
  
  • Known Issue Overview
  • Known Issues Details
    
    • DGX A800 Station/Server: mig-parted config
    • Regression of CUDA application startup performance
    • NVSM Stress Test Logs Do Not Contain Summary Information
    • nvidia-release-upgrade May Report That Not All Updates Have Been Installed and Exit
    • Duplicate EFI Variable May Cause efibootmgr to Fail
    • Erroneous Insufficient Power Error May Occur for PCIe Slots
    • AMD Crypto Coprocessor is not Supported
    • nvsm show alerts Reports NVSwitch PCIe Link Width is Degraded
    • nvsm show health Reports Firmware as Not Authenticated
    • Running NGC Containers Older than 20.10 May Produce “Incompatible MOFED Driver” Message
    • System May Slow Down When Using mpirun
    • Forced Reboot Hangs the OS
    • Applications that call the cuCTXCreate API Might Experience a Performance Drop
    • NVIDIA Desktop Shortcuts Not Updated After a DGX OS Release Upgrade
    • Unable to Set a Separate/xinerama Mode through the xorg.conf File or through nvidia-settings
  • Resolved Issues Details
    
    • NVSM Platform Displays as Unsupported
    • NVSM Enumerates NVSwitches as 8-13 Instead of 0-5
    • A System with Encrypted rootfs May Fail to Boot if one of the M.2 drives is Corrupted
    • NVSM Fails to Show CPU Information on Non-English Locales
    • Driver Version Mismatch Reported
  • Known Limitations Details
    
    • No RAID Partition Created After ISO Install
    • System Services Startup Messages Appear Upon Completion of First-Boot Setup
    • [DGX A100]: Hot-plugging of Storage Drives not Supported
    • [DGX A100]: Syslog Contains Numerous “SM LID is 0, maybe no SM is running” Error Messages
    • [DGX-2]: Serial Over LAN Does not Work After Cold Resetting the BMC
    • [DGX-2]: Some BMC Dashboard Quick Links Appear Erroneously
    • [DGX-2]: Applications Cannot be Run Immediately Upon Powering on the DGX-2
    • [DGX-1]: Script Cannot Recreate RAID Array After Re-inserting a Known Good SSD
    • [DGX Station A100] Suspend and Power Button Section Appears in Power Settings
    • [DGX-2] NVSM Does not Detect Downgraded GPU PCIe Link

Appendices

• Appendix A: DGX OS Connectivity Requirements
  
  • In-Band Management, Storage, and Compute Networks
  • Out-of-Band Management
• Appendix B: Downgrade Firmware for Mellanox ConnectX-4 Cards
  
  • Checking the Device Type
  • Downgrading the Firmware
• Appendix C: DGX Software Stack
  
  • NVIDIA DGX Software Packages
  • DGX Kernel Parameters
• Appendix D: Base OS 5.0 PXE Boot Setup
  
  • Overview of the PXE Server
    
    • Setting Up dnsmasq for DHCP
    • Configure the FTP Side
    • Configuring the HTTP File Directory and ISO Image
    • Configure your DHCP Server
    • Configuring the syslinux Bootloader
    • Live Boot Parameters
    • Setup GRUB PXE Server
    • Optional: Configure the NVIDIA ConnextX cards to PXE boot
    • Query the UEFI PXE ROM State
    • Enable the UEFI PXE ROM
    • Curtin Customizations
    • Customizing the Curtin YAML
      
      • NVIDIA-Specific Curtin Variables
      • CHANGE_STORAGE_REG
      • CHANGE_STORAGE_ENC
      • CHANGE_BOOT_DISK_NAME_x
    • Boot the DGX System over PXE
    • Reference: Other IPMI Boot Options
• Appendix E: Installing DGX OS on Air-Gapped Systems
  
  • Creating a Local Mirror of the NVIDIA and Canonical Repositories
  • Creating the Mirror of the Repositories
  • Configuring the Target Air-Gapped System

Misc

• Third-Party License Notices
  
  • Micron msecli
  • Mellanox (OFED)
• Notices
  
  • Notice
  • Trademarks
  • Copyright