Version EL8-23.08

Attention

If your system is running a version earlier than EL8-22.05, you need to update the keys on the system. Refer to Rotating the GPG Key for more information about how to rotate the keys.

The DGX Software for Red Hat Enterprise Linux 8 and Rocky Linux 8, EL8-23.08, is available.

EL8-23.08 supports all DGX products - DGX H100, DGX A100/A800, DGX-2, DGX-1, DGX Station, DGX Station A100, and DGX Station A800.

Important

Installing or updating to EL8-23.08 also updates the installed Red Hat Enterprise Linux 8 distribution to the latest version.

  • NVIDIA GPUDirect Storage (GDS) v1.1 supports Red Hat Enterprise Linux 8.8.

  • If you need to use the Mellanox OpenFabrics Enterprise Distribution for Linux (MLNX_OFED), before you install or update to EL8-23.08, ensure that there is a MLNX_OFED package version available that supports the latest Red Hat Enterprise Linux 8 version.

Update September 2023

  • NVIDIA Drivers:

  • CUDA Toolkit: 12.2

  • cuDNN: 8.9.4.25

  • DCGM: 3.1.8

  • NVSM: 23.06.04

  • NCCL: 2.18.5

  • Docker: 23.0.6

  • DLFW: 23.05

  • NVIDIA Container Toolkit:

    • nvidia-docker2 2.13.0-1

    • libnvidia-container-tools 1.13.4-1

    • libnvidia-container1 1.13.4-1

    • nvidia-container-toolkit 1.13.4-1

Software Contents

The following table provides version information for software included in the DGX Software Stack for Red Hat Enterprise Linux 8 and Rocky Linux 8.8.

Note

Unlike the DGX OS shipped with the NVIDIA DGX system, the DGX software stack for Red Hat does not include the Mellanox OpenFabrics Enterprise Distribution (MLNX_OFED) for Linux. When using MLNX_OFED with Red Hat, ensure you install a supported MLNX_OFED kernel version to avoid incompatibilities with the Red Hat distribution kernel.

Refer to the DGX Software for Red Hat Enterprise Linux 8 Installation Guide for instructions.

Contents of the Repositories

Component

Version

Additional Information

OS

Red Hat Enterprise Linux 8.8 and Rocky Linux 8.8

Kernel

4.18.0-477.10.1.el8_8

GPU Driver

535.104.05 (CUDA Toolkit 12.2)

525.125.06 (CUDA Toolkit 12.0)

515.105.01 (CUDA Toolkit 11.7)

470.199.02 (CUDA Toolkit 11.4)

450.248.02 (CUDA Toolkit 11.0)

Refer to the NVIDIA Data Center GPU documentation

CUDA Toolkit

12.2

Note: The CUDA Toolkit is only installed for DGX Stations and option for DGX servers. Refer also to the latest CUDA Release Notes for driver compatibility information.

NCCL

2.18.5

cuDNN

8.9.4.25

DCGM

3.1.8

NVSM

23.06.04

Refer to NVIDIA System Management Documentation

Docker Engine

23.0.6

Refer to Docker Engine

DLFW

23.05

NGC CLI

3.17.0

Refer to NGC CLI Documentation

NVIDIA Container Toolkit

1.13

NVIDIA Container Toolkit includes the following packages:

  • libnvidia-container-tools: 1.13.4-1

  • libnvidia-container1: 1.13.4-1

  • nvidia-container-toolkit: 1.13.4-1

  • nvidia-docker2: 2.13.0-1

GPUDirect Storage (GDS)

1.0

Refer to GDS Documentation

MIG Configuration Tool

nvidia-mig-manager 0.4.3

Refer to NVIDIA mig-parted github pages: and deployments

nvipmitool

1.0.6.0

nvidia-peer-memory

nvidia-peer-memory-dkms

1.3.0

The following table provides information about the supported OS and matching firmware versions for Mellanox OFED.

OS

DGX-1, DGX-2

ConnectX-4 or ConnectX-5

DGX A100

ConnectX-6 (CX-6)

DGX A100

ConnectX-7 (CX-7)

DGX H100

ConnectX-7 (CX-7)

RHEL 8

5.8-3.0.7.0

  • CX-5: 16.35.3006

  • CX-4: 12.28.2006

  • RHEL 8.8

5.8-3.0.7.0

  • CX-6: 20.35.3006

  • RHEL 8.8

5.4-3.7.5.0

  • CX-7: 28.34.4000

  • RHEL 8.8

5.9-0.5.6.0.127

  • CX-7: 28.36.2050

  • RHEL 8.7

The drivers are compatible with RHEL 8.8. Refer to DGX Software for Red Hat Enterprise Linux 8 Installation Guide for installation information.

Note

For information about LTS software versions for related networking components, refer to the Networking Long-Term Support Releases page.

Compatibility

NVIDIA has validated and tested DGX Software version EL8-23.08 on the following systems:

  • Linux Distribution and kernel:

    • Red Hat Enterprise Linux 8.8

    • Rocky Linux 8

    • Kernel 4.18.0-477.10.1

  • NVIDIA DGX systems

    • NVIDIA DGX H100 with Red Hat Enterprise Linux 8.8

    • NVIDIA DGX A100/A800 with Red Hat Enterprise Linux 8.6 and Rocky Linux 8

    • NVIDIA DGX-2 with Red Hat Enterprise Linux 8.6 and Rocky Linux 8

    • NVIDIA DGX-1 (V100) with Red Hat Enterprise Linux 8.6 and Rocky Linux 8

    • NVIDIA DGX Station with Red Hat Enterprise Linux 8.6 and Rocky Linux 8

    • NVIDIA DGX Station A100 with Red Hat Enterprise Linux 8.6 and Rocky Linux 8

    • NVIDIA DGX Station A800 with Red Hat Enterprise Linux 8.6 and Rocky Linux 8

  • 22.08 Deep Learning Framework containers

  • NVIDIA GPUDirect Storage v1.0 - refer to the GDS documentation for additional information.

  • MLNX OFED version 5.4-3.5.8.0

  • ConnectX Firmware: see table above

Update Instructions

See the section Installing and Updating the Software for instructions.