DGX OS 4 Server Software Release Notes

The DGX OS 4 Server Release Notes.

1. Abstract

The NVIDIA® DGX™ servers come with a DGX OS software image. This base operating system on DGX OS servers allows customers to use their own on-site scheduling and management software to run GPU-accelerated applications in Docker containers provided by NVIDIA and in customized Docker containers.

2. Rotating the GPG Keys

NVIDIA constantly evaluates and improves security implementations. As part of these improvements, we are rolling out changes to harden the security and reliability of our repositories. These changes require rotating the GPG keys that are used to sign the metadata and packages in those repositories. This section provides information about how to rotate the GPG keys on your system.

  1. Download the new repository setup packages.
    wget https://international.download.nvidia.com/dgx/repos/bionic/pool/multiverse/d/dgx-repo/dgx-repo_1.0-5_amd64.deb
wget https://international.download.nvidia.com/dgx/repos/bionic/pool/multiverse/n/nvidia-repo-keys/nvidia-repo-keys_22.04-1_all.deb
  2. Directly install the .deb packages, which skips the GPG check performed in apt.

    If prompted, ensure that you accept the maintainer’s version for all files. 

    $ sudo dpkg --force-confnew -i ./nvidia-repo-keys_22.04-1_all.deb  ./dgx-repo_1.0-5_amd64.deb
  3. Manually revoke the previous DGX and CUDA GPG keys.
    $ sudo apt-key del 629C85F2
$ sudo apt-key del 7FA2AF80
  4. Update optional repositories that were previously enabled.
    for x in $(find /etc/apt/sources.list.d -name "dgx*.list"); do if ! grep -q "signed-by" $x; then sudo sed -i 's|^deb |deb [arch=amd64 signed-by=/usr/share/keyrings/dgx_debian_prod.gpg] |' $x; fi; done

3. DGX OS Server Releases and Versioning

This information helps you understand the DGX OS Server release numbering convention and your options to upgrade your DGX OS Server software.

DGX OS Server Releases

DGX OS is a customized Linux distribution that is based on Ubuntu Linux. It includes platform-specific configurations, diagnostic and monitoring tools, and the drivers that are required to provide the stable, tested, and supported OS to run AI, machine learning, and analytics applications on DGX systems.

DGX OS is released twice a year, typically around February and August, for two years after the first release. Updates are provided between releases and thereafter for the entire support duration.

Release Versions

The DGX OS release numbering convention is MAJOR.MINOR, and it defines the following types of releases:

  • Major releases are typically based on Ubuntu releases, which include new kernel versions and new features that are not always backwards compatible.
    For example:
    • DGX OS 5.x releases are based on Ubuntu 20.04.
    • DGX OS 4.x is based on Ubuntu 18.04.
  • Minor releases include mostly new NVIDIA features and accumulated bug fixes and security updates.

    These releases are incremental and always include all previous software changes.

    • In DGX OS 4 and earlier, minor releases were also typically aligned with NVIDIA Graphics Drivers for Linux releases.
    • In DGX OS 5, you now have the option to install newer NVIDIA Graphic Drivers independently of the DGX OS release.

DGX OS Release Mechanisms

This section provides information about the DGX OS release mechanisms that are available to install or upgrade to the latest version of the DGX OS.

The ISO Image

DGX OS is released as an ISO image that includes the necessary packages and an autonomous installer. Updated versions of the ISO image are also released that:

  • Provide bug fixes and security mitigations.

  • Improve the installation experience.

  • Provide hardware configuration support.

You should always use the latest ISO image, except when you need to restore the system to an earlier version.

Warning: This image allows you to install or reimage a DGX system to restore the system to a default state, but the process erases all of the changes that you applied to the OS.

The Linux Software Repositories

Upgrades to DGX OS are provided through the software repositories. Software repositories are storage locations from which your system retrieves and installs OS updates and applications. The repositories used by DGX OS are hosted by Canonical for the Ubuntu OS and NVIDIA for DGX specific software and other NVIDIA software. Each repository is a collection of software packages that are intended to install additional software and to update the software on DGX systems.

New versions of these packages, which contain bug fixes and security updates, provide an update to DGX OS releases. The repositories are also updated to include hardware enablement, which might add support for a new system or a new hardware component, such as a network card or disk drive. This update does not affect existing hardware configurations.

System upgrades are cumulative, which means that your systems will always receive the latest version of all of the updated software components. You cannot select which upgrades to make or limit upgrades to the non-latest DGX OS 5.x release.

Important: We recommend that you do not update only individual components.

Before you update a system, refer to the DGX OS Software Release Notes for a list of the available updates. For more information on displaying available updates and upgrade instructions, refer to the DGX OS 5 User Guide.

4. Current NVIDIA DGX OS 4 Server Software

This section lists the NVIDIA® DGX™ OS 4 Server software images and updates that have been released for the NVIDIA DGX-1, and DGX-2 systems. To determine the version of the DGX OS Server software on your system, enter the following command. 

$ grep VERSION /etc/dgx-release

This will show the version installed on the system from the ISO image (DGX_SWBUILD_VERSION) and any subsequent updates performed on the system (DGX_OTA_VERSION). The latest version listed is the currently installed version.

Release 4.14 (DGX-1, DGX-2)

Attention: If your system is running a version earlier than DGX OS 4.12, you need to update the keys on the system. Refer to Rotating the GPG Key for more information about how to rotate the keys
  • Highlights
    • Updated the NVIDIA Release 450 GPU driver to 450.203.03
    • Updated the NVSM to 20.09.40
    • Updated DCGM software to version 2.4.5
    • Docker updated to version 20.10.17
  • Initial release: 4.14.0

    See the following for additional changes, update instructions, and known issues.

    DGX OS Server 4.14 Release Notes

Release 4.13 (DGX-1, DGX-2)

Attention: If your system is running a version earlier than DGX OS 4.12, you need to update the keys on the system. Refer to Rotating the GPG Key for more information about how to rotate the keys
  • Highlights
    • Updated the NVIDIA Release 450 GPU driver to 450.191.01.
    • Updated the NVSM to 20.09.37
  • Initial release: 4.13.0

    See the following for additional changes, update instructions, and known issues.

    DGX OS Server 4.13 Release Notes

Release 4.12 (DGX-1, DGX-2)

Important: In DGX OS 4.12, the GPG keys that are used to sign the metadata and packages in those repositories need to be rotated. Refer to Rotating the GPG Keys for more information.

None of the component versions have changed in DGX OS 4.12.

Release 4.11 (DGX-1, DGX-2 only)

Release 4.10 (DGX-1, DGX-2 only)

Release 4.9 (DGX-1, DGX-2 only)

  • Highlights
    • Updated the NVIDIA Release 418 GPU driver to 418.211.00.
    • Updated the NVIDIA Release 450 GPU driver to 450.142.00
    • Improved process for updating the ConnectX card firmware.

      If updating from DGX OS 4.8 or later, the firmware for all cards are now updated in parallel instead of one at a time, significantly reducing the time to update all cards.

  • Initial release: 4.9.0

    See the following for additional changes, update instructions, and known issues.

    DGX OS Server 4.9 Release Notes

Release 4.8 (DGX-1, DGX-2 only)

  • Highlights
    • Updated the NVIDIA Release 418 GPU driver to 418.197.02.
    • Updated the NVIDIA Release 450 GPU driver to 450.119.04
  • Initial release: 4.8.0

    See the following for update instructions, additional information, and known issues.

    DGX OS Server 4.8 Release Notes

Release 4.7 (DGX-1, DGX-2 only)

  • Highlights
    • Updated the NVIDIA Release 418 GPU driver to 418.181.07.
    • Updated the NVIDIA Release 450 GPU driver to 450.102.04
    • Updated NVIDIA KVM sotware
  • Initial release: 4.7.0

    See the following for update instructions, additional information, and known issues.

    DGX OS Server 4.7 Release Notes

Release 4.6 (DGX-1, DGX-2 only)

  • Highlights
    • Updated the NVIDIA Release 418 GPU driver to 418.165.02.
    • Updated the NVIDIA Release 450 GPU driver to 450.80.02
    • Updated the Mellanox OFED driver to 4.9
  • Initial release: 4.6.0

    See the following for update instructions, additional information, and known issues.

    DGX OS Server 4.6 Release Notes

Release 4.5 (DGX-1, DGX-2 only)

Important: Updates to Release 4.5 are no longer available. Attempts to update to Release 4.5 will result in upgrading your DGX Server software to the latest release of DGX OS Server 4.
  • Highlights
    • Updated NVIDIA GPU driver to 418.152.00.
    • Added optional repository to install the NVIDIA Release 450 GPU driver
    • Updated to Ubuntu 18.04.4 LTS
  • Initial release: 4.5.0

    See the following for update instructions, additional information, and known issues.

    DGX OS Server 4.5 Release Notes

Release 4.4 (DGX-1, DGX-2 only)

Important: Updates to Release 4.4 are no longer available. Attempts to update to Release 4.4 will result in upgrading your DGX Server software to the latest release of DGX OS Server 4.
  • Highlights
    • Updated NVIDIA GPU driver to 418.126.02.
    • Updated NVSM software to version 20.01.15
    • Added the ability to disable and re-enable CPU mitigations.
    • Added support for PXE boot.
    • Added support for Mellanox CX6 cards on DGX-1 (Tesla V100)
  • Latest patch update: 4.4.1
  • See the following for update instructions, additional information, and known issues.

Release 4.3 (DGX-1, DGX-2 only)

Important: Updates to Release 4.3 are no longer available. Attempts to update to Release 4.3 will result in upgrading your DGX Server software to the latest release of DGX OS Server 4.
  • Highlights
    • Updated NVIDIA GPU driver to 418.116.00.
    • Updated NVSM software to version 19.08.6
    • Updated DCGM software to version 1.7.2
    • Docker updated to version 19.03.4-ce
  • See the following for update instructions, additional information, and known issues.

Release 4.2 (DGX-1, DGX-2 only)

Important: Updates to Release 4.2 are no longer available. Attempts to update to Release 4.2 will result in upgrading your DGX Server software to the latest release of DGX OS Server 4.
  • Highlights
    • Updated NVSM software to version 19.08
    • Docker update to version 19.03-ce
    • MLNX_OFED 4.6
  • See the following for update instructions, additional information, and known issues.

Release 4.1 (DGX-1, DGX-2 only)

Important: Updates to Release 4.1 are no longer available. Attempts to update to Release 4.1 will result in upgrading your DGX Server software to the latest release of DGX OS Server 4.
  • Highlights
    • NVIDIA GPU Driver Release 418
      • Supports CUDA 10.1
  • Latest patch update: 4.1.1
  • See the following for update instructions, additional information, and known issues.

Release 4.0 (DGX-1, DGX-2 only)

Important: Updates to Release 4.0 are no longer available. Attempts to update to Release 4.0 will result in upgrading your DGX Server software to the latest release of DGX OS Server 4.

5. Deprecated NVIDIA DGX OS Server Releases

This section lists the NVIDIA® DGX™ OS Server software images and updates that have been deprecated.

Release 4.99 (DGX A100 Only)

  • Highlights
    • Supports the NVIDIA DGX A100 server only.
    • Updated NVIDIA GPU driver to Release 450.
  • Latest update: 4.99.11

    See the following for update instructions, additional information, and known issues.

    DGX OS Server 4.99.x Release Notes

Release 3.1

  • Highlights
    • Ubuntu 16.04 LTS
      • Initialization daemon changed from Upstart to systemd.
      • Updated network interface naming policy.

        Policy now uses predictable names, rather than the native naming scheme used in previous releases. The first and second Ethernet interfaces, enumerated as em1 and em2 in previous releases, will now enumerate as enp1s0f0 and enp1s0f1 respectively.

    • NVIDIA GPU Driver Release 384
      • Supports the NVIDIA Tesla™ V100 GPUs.
      • Supports CUDA 9.0 .
    • CUDA drivers and diagnostic packages updated to Release 384.
    • Mellanox drivers updated to 4.0.
    • Docker CE and nvidia-docker are pre-installed, and the docker daemon automatically launched.
  • See the following for additional information, known issues, and update instructions for specific versions.

Release 2.1

  • End of Support Notice

    NVIDIA will discontinue support for DGX OS Server 2.1 effective December 31, 2018. After this date, NVIDIA will no longer provide software updates for this release. You may need to upgrade your DGX-1 systems to a supported release to address issues or security vulnerabilities.

    Evaluate your installation to determine update needs and strategy. Systems can be updated now to DGX OS Server 3.1 (which includes Ubuntu 16.04), or updated later to update to a future software release that includes Ubuntu 18.04.

  • Highlights
    • Ubuntu 14.04 LTS

      Updates installed packages with the latest versions available at the time of update.

    • NVIDIA GPU Driver Release 384
      • Supports CUDA 9.0
    • CUDA drivers and diagnostic packages updated to Release 384.
    • CUDA Toolkit (if already installed) will get updated to 9.0.176.
  • See the following for additional information and update instructions for specific versions.

Release 2.0

  • Highlights
    • Ubuntu 14.04 LTS
    • NVIDIA GPU Driver Release 375
  • End of Support Notice

    NVIDIA will discontinue support for DGX OS Server 2.0 effective December 31, 2018. After this date, NVIDIA will no longer provide software updates for this release. You may need to upgrade your DGX-1 systems to a supported release to address issues or security vulnerabilities.

    Evaluate your installation to determine update needs and strategy. Systems can be updated now to DGX OS Server 3.1 (which includes Ubuntu 16.04), or updated later to update to a future software release that includes Ubuntu 18.04.

  • See the following for additional information and update instructions for specific versions. .

6. Third Party License Notice

This NVIDIA product contains third party software that is being made available to you under their respective open source software licenses. Some of those licenses also require specific legal information to be included in the product. This section provides such information.

msceli

The msecli utility (https://www.micron.com/products/solid-state-storage/storage-executive-software) is provided under the following terms: 

Micron Technology, Inc. Software License Agreement   

PLEASE READ THIS LICENSE AGREEMENT ("AGREEMENT") FROM MICRON 
TECHNOLOGY, INC. ("MTI") CAREFULLY: BY INSTALLING, COPYING OR 
OTHERWISE USING THIS SOFTWARE AND ANY RELATED PRINTED MATERIALS 
("SOFTWARE"), YOU ARE ACCEPTING AND AGREEING TO THE TERMS OF 
THIS AGREEMENT. IF YOU DO NOT AGREE WITH THE TERMS OF THIS 
AGREEMENT, DO NOT INSTALL THE SOFTWARE.   

LICENSE: MTI hereby grants to you the following rights: You may 
use and make one (1) backup copy the Software subject to the 
terms of this Agreement. You must maintain all copyright notices 
on all copies of the Software. You agree not to modify, adapt, 
decompile, reverse engineer, disassemble, or otherwise translate 
the Software. MTI may make changes to the Software at any time 
without notice to you. In addition MTI is under no obligation 
whatsoever to update, maintain, or provide new versions or other 
support for the Software. 

OWNERSHIP OF MATERIALS: You acknowledge and agree that the 
Software is proprietary property of MTI 
(and/or its licensors) and is protected by United States 
copyright law and international treaty provisions. Except as 
expressly provided herein, MTI does not grant any express or 
implied right to you under any patents, copyrights, trademarks, 
or trade secret information. You further acknowledge and agree 
that all right, title, and interest in and to the Software, 
including associated proprietary rights, are and shall remain 
with MTI (and/or its licensors). This Agreement does not convey 
to you an interest in or to the Software, but only a limited 
right to use and copy the Software in accordance with the terms 
of this Agreement. The Software is licensed to you and not sold. 

DISCLAIMER OF WARRANTY: THE SOFTWARE IS PROVIDED "AS IS" WITHOUT 
WARRANTY OF ANY KIND. MTI EXPRESSLY DISCLAIMS ALL WARRANTIES 
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO, 
NONINFRINGEMENT OF THIRD PARTY RIGHTS, AND ANY IMPLIED 
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR 
PURPOSE. MTI DOES NOT WARRANT THAT THE SOFTWARE WILL MEET YOUR 
REQUIREMENTS, OR THAT THE OPERATION OF THE SOFTWARE WILL BE 
UNINTERRUPTED OR ERROR-FREE. FURTHERMORE, MTI DOES NOT MAKE ANY 
REPRESENTATIONS REGARDING THE USE OR THE RESULTS OF THE USE OF 
THE SOFTWARE IN TERMS OF ITS CORRECTNESS, ACCURACY, RELIABILITY, 
OR OTHERWISE. THE ENTIRE RISK ARISING OUT OF USE OR PERFORMANCE 
OF THE SOFTWARE REMAINS WITH YOU. IN NO EVENT SHALL MTI, ITS 
AFFILIATED COMPANIES OR THEIR SUPPLIERS BE LIABLE FOR ANY DIRECT,
INDIRECT, CONSEQUENTIAL, INCIDENTAL, OR SPECIAL DAMAGES 
(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, 
BUSINESS INTERRUPTION, OR LOSS OF INFORMATION) ARISING OUT OF 
YOUR USE OF OR INABILITY TO USE THE SOFTWARE, EVEN IF MTI HAS 
BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Because some 
jurisdictions prohibit the exclusion or limitation of liability 
for consequential or incidental damages, the above limitation 
may not apply to you.   

TERMINATION OF THIS LICENSE: MTI may terminate this license at 
any time if you are in breach of any of the terms of this 
Agreement. Upon termination, you will immediately destroy all 
copies the Software.   

GENERAL: This Agreement constitutes the entire agreement between 
MTI and you regarding the subject matter hereof and supersedes 
all previous oral or written communications between the parties. 
This Agreement shall be governed by the laws of the State of 
Idaho without regard to its conflict of laws rules.   

CONTACT: If you have any questions about the terms of this 
Agreement, please contact MTI's legal department at 
(208) 368-4500. By proceeding with the installation of the 
Software, you agree to the terms of this Agreement. You must 
agree to the terms in order to install and use the Software.

Mellanox (OFED)

MLNX OFED (http://www.mellanox.com/) version 3.3-1.0.4.0 is provided under the following terms: 

Copyright (c) 2006 Mellanox Technologies.
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, 
are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, 
this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, 
this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, 
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, 
OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, 
OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 
POSSIBILITY OF SUCH DAMAGE.

Notices

Notice

THE INFORMATION IN THIS DOCUMENT AND ALL OTHER INFORMATION CONTAINED IN NVIDIA DOCUMENTATION REFERENCED IN THIS DOCUMENT IS PROVIDED “AS IS.” NVIDIA MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE INFORMATION FOR THE PRODUCT, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE. Notwithstanding any damages that customer might incur for any reason whatsoever, NVIDIA’s aggregate and cumulative liability towards customer for the product described in this documentshall be limited in accordance with the NVIDIA terms and conditions of sale for the product.

THE NVIDIA PRODUCT DESCRIBED IN THIS DOCUMENT IS NOT FAULT TOLERANT AND IS NOT DESIGNED, MANUFACTURED OR INTENDED FOR USE IN CONNECTION WITH THE DESIGN, CONSTRUCTION, MAINTENANCE, AND/OR OPERATION OF ANY SYSTEM WHERE THE USE OR A FAILURE OF SUCH SYSTEM COULD RESULT IN A SITUATION THAT THREATENS THE SAFETY OF HUMAN LIFE OR SEVERE PHYSICAL HARM OR PROPERTY DAMAGE (INCLUDING, FOR EXAMPLE, USE IN CONNECTION WITH ANY NUCLEAR, AVIONICS, LIFE SUPPORT OR OTHER LIFE CRITICAL APPLICATION). NVIDIA EXPRESSLY DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY OF FITNESS FOR SUCH HIGH RISK USES. NVIDIA SHALL NOT BE LIABLE TO CUSTOMER OR ANY THIRD PARTY, IN WHOLE OR IN PART, FOR ANY CLAIMS OR DAMAGES ARISING FROM SUCH HIGH RISK USES.

NVIDIA makes no representation or warranty that the product described in this documentwill be suitable for any specified use without further testing or modification. Testing of all parameters of each product is not necessarily performed by NVIDIA. It is customer’s sole responsibility to ensure the product is suitable and fit for the application planned by customer and to do the necessary testing for the application in order to avoid a default of the application or the product. Weaknesses in customer’s product designs may affect the quality and reliability of the NVIDIA product and may result in additional or different conditions and/or requirements beyond those contained in this document. NVIDIA does not accept any liability related to any default, damage, costs or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document, or (ii) customer product designs.

Other than the right for customer to use the information in this document with the product, no other license, either expressed or implied, is hereby granted by NVIDIA under this document. Reproduction of information in this document is permissible only if reproduction is approved by NVIDIA in writing, is reproduced without alteration, and is accompanied by all associated conditions, limitations, and notices.

Trademarks

NVIDIA, the NVIDIA logo, DGX, DGX-1, DGX-2, DGX A100, DGX Station, and DGX Station A100 are trademarks and/or registered trademarks of NVIDIA Corporation in the Unites States and other countries. Other company and product names may be trademarks of the respective companies with which they are associated.