Appendix D: Base OS 5.0 PXE Boot Setup

Base OS - DGX OS 5

The DGX-Server UEFI BIOS supports PXE boot. Several manual customization steps are required to get PXE to boot the Base OS image.

Caution

This document is meant to be used as a reference. Explicit instructions are not given to configure the DHCP, FTP, and TFTP servers. It is expected that the end user’s IT team will configure them to fit within their company’s security guidelines.

Prerequesites

  • A TFTP and FTP server are setup

    • IP address is <FTP IP>

    • Fully qualified host is <FTP host>

This document is intended to provide detailed step-by-step instructions on how to set up a PXE boot environment for DGX systems. The examples are based on a DGX A100. There are several major components of the solution:

  • DHCP server: dnsmasq (https://thekelleys.org.uk/dnsmasq/doc.html) is used in this doc.

  • TFTP server: dnsmasq is also used as a TFTP server.

  • HTTP server: HTTP server is used to transfer large files, such is ISO image and initrd. Alternatively, FTP can also be used for this purpose. HTTP in used this doc.

  • Syslinux: Linux bootloader software package. section

The PXE Server is divided up into the following areas:

  • Bootloader (syslinux)

  • FTP contents (i.e. the ISO image and its extracted contents)

Note that when comparing the Base OS 5.x contents to previous versions, there are comparatively very few files in the ISO. The rough directory structure on the FTP server will look like (where 5.x.y is the version you’re installing)

Copy
Copied!
            

/local/ftp └── dgxbaseos-5.x.y ├── base_os_5.x.y.iso ├── boot/ │ └── grub │ └── bios.img │ └── boot.catdefault │ └── grub.cfg └── EFI │ └── efiboot.imgg ├── initrd ├── live/ │ └── filesystem.squashfs ├── md5sum.txt ├── UBUNTU_LIVE ├── vmlinuz /local/syslinux └── efi64 ├── syslinux.efi ├── ldlinux.e64 ├── pxelinux.cfg │ └── default ├── sdgx-5.x.y/ │ └── vmlinuz │ └── initrd

The tftp-server (controlled by the xinetd service and configuration found in /etc/xinetd.d/tftp) points to the syslinux directory for when the system PXE boots. TFTP is what transfers the syslinux.efi file that is designated in the DHCP server’s dhcpd.conf file (see Configure your DHCP Server). By default after the syslinux.efi is booted it looks for a pxelinux.cfg/default file with the menu options for booting further, i.e. the kernel and initrd in that sdgx/ directory. That config file will look in the same working directory for its files as it did for the pxelinux.cfg directory. In this case it is /local/syslinux/efi64.

The following steps will assume the DHCP and PXE servers are configured to use the above directory structure. The lab admin, or whoever is in charge of deploying the PXE environment, should change the directory names and structure to fit their infrastructure.

Setting Up dnsmasq for DHCP

The DGX system BIOS does not allow you to manually specify a PXE server. You need to use the DHCP server to determine what it is and from which file to boot.

  1. Install dnsmasq.

    Copy
    Copied!
                

    $ apt install dnsmasq

  2. Back up the config file prior to customizing it.

    Copy
    Copied!
                

    $ mv /etc/dnsmasq.conf /etc/dnsmasq.conf.backup

  3. Edit the config file using the appropriate values for your system.

    Copy
    Copied!
                

    $ vi /etc/dnsmasq.conf

    Copy
    Copied!
                

    interface=enp226s0 bind-interfaces domain=nvidia.com dhcp-range=enp226s0,10.33.254.106,static,1h dhcp-host=5c:ff:35:e1:a4:d1,DGX-A100-PM2,10.33.254.106 dhcp-option=option:router,10.33.254.1 dhcp-option=option:dns-server,8.8.8.8 enable-tftp tftp-root=/local/syslinux/efi64/ dhcp-boot=syslinux.efi

  4. Create and set up the local/syslinux/efi64/ folder.

    Copy
    Copied!
                

    $ sudo mkdir -p /local/syslinux/efi64

    Copy
    Copied!
                

    $ systemctl restart dnsmasq

    Copy
    Copied!
                

    $ systemctl status dnsmasq

XXXXXXXXXXXXX

Configure the FTP Side

Mount the ISO

Assume you have a copy of the 5.x.y ISO located at “/tmp/base_os_5.x.y.iso”, and your mount point is “/mnt”:

Copy
Copied!
            

$ sudo mount -o loop /tmp/base_os_5.x.y.iso /mnt

Copy the ISO and its extracted contents to the FTP Directory

Using the sample directory structure in the Overview, the FTP root directory is “/local/ftp/”, and there is a “dgxbaseos-5.x.y/” directory under it.

Copy
Copied!
            

$ cp -r /mnt/* /local/ftp/dgxbaseos-5.x.y/

Copy
Copied!
            

$ cp /tmp/base_os_5.x.y.iso /local/ftp/dgxbaseos-5.x.y/

Ensure all the files are owned, or are at least readable, by the “ftp” user. It’s now safe to unmount the ISO if you want to:

Copy
Copied!
            

$ sudo umount /mnt

Configuring the HTTP File Directory and ISO Image

Here are the steps to configure the http file directory and ISO image for PXE boot.

The instructions produce the following file structure on the PXE server.

Copy
Copied!
            

/local/http └── dgxbaseos-5.x.y ├── base_os_5.x.y.iso ├── initrd └── vmlinuz

You should have obtained and copied the ISO image to “/tmp/dgxbaseos-5.x.y.iso”.

  1. Create the DGX OS directory.

    Copy
    Copied!
                

    $ sudo mkdir -p /local/http/dgxbaseos-5.x.y

  2. Mount the ISO.

    Your mount point is /mnt.

    Copy
    Copied!
                

    $ sudo mount -o loop /tmp/base_os_5.x.y.iso /mnt

  3. Copy the ISO and its extracted contents to the HTTP directory.

    Copy
    Copied!
                

    $ sudo cp /mnt/{initrd,vmlinuz} /local/http/dgxbaseos-5.x.y/

    Copy
    Copied!
                

    $ cp /tmp/dgxbaseos-5.0.2.iso /local/http/dgxbaseos-5.x.y/

  4. Ensure all the files are owned, or are at least readable, by the http user.

  5. You can now unmount the ISO.

    Copy
    Copied!
                

    $ sudo umount /mnt

Configure your DHCP Server

The DGX-Server BIOS unfortunately does not allow you to specify a PXE server manually. You have to rely on the DHCP server to tell you what it is, and what file to boot from. In the DHCP server’s /etc/dhcp/dhcpd.conf file:

  • Set the “next-server” to <FTP IP_ADDR>

  • Set the “filename” to “efi64/syslinux.efi”

Again, this assumes the directory structure in the Overview.

Optional: Configure the NVIDIA ConnextX cards to PXE boot

DGX-Servers may also PXE boot using the NVIDIA ConnextX cards. If you are logged into the DGX system, you can perform this section’s steps using the /usr/sbin/mlnx_pxe_setup.bash tool, which will enable the UEFI PXE ROM of every NVIDIA ConnectX InfiniBand device found.

Otherwise, proceed with the manual steps below.

Query UEFI PXE ROM state

In order to PXE boot from the MLNX CX-4/5/6 cards, you must first enable the UEFI PXE ROM of the card you wish to PXE boot from because it is disabled by default. This needs to be performed from the DGX Server host OS itself, it can’t be done remotely. The configuration of the NVIDIA ConnextX devices is handled through the “mlxconfig” command.

To determine the device name and current configurations of the NVIDIA ConnectX cards, run $ sudo mlxconfig query

Copy
Copied!
            

$ sudo mlxconfig query Device #1: ---------- Device type: ConnectX4 Name: MCX455A-ECA_Ax Description: ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6 Device: /dev/mst/mt4115_pciconf3 Configurations: Next Boot ... ... EXP_ROM_UEFI_x86_ENABLE False(0) ... ... Enable UEFI PXE ROM

The EXP_ROM_UEFI_x86_ENABLE configuration must be set to True(1) for the NVIDIA ConnectX card that you wish to PXE boot from, and reboot.

Copy
Copied!
            

$ sudo mlxconfig -y -d /dev/mst/mt4115_pciconf3 set EXP_ROM_UEFI_x86_ENABLE=1

Reboot the system:

Copy
Copied!
            

$ sudo reboot

Upon reboot, confirm the configuration was set.

Copy
Copied!
            

$ sudo mlxconfig query Device #1: ---------- Device type: ConnectX4 Name: MCX455A-ECA_Ax Description: ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6 Device: /dev/mst/mt4115_pciconf3 Configurations: Next Boot ... ... EXP_ROM_UEFI_x86_ENABLE True(1) ... ...

Configuring the syslinux Bootloader

These instructions install the syslinux 6.04-pre1 package. The 5.x.y OS image does not include a bootloader for PXE boot. Instead, the syslinux from kernel.org is used. The syslinux 6.04-pre1 package is installed because fixes in 6.04 are required to support PXE booting from the Mellanox ethernet cards as an option. PXE boot from the Mellanox cards also require at least 16.23.1020 level firmware with:

  • FW 16.23.1020

  • PXE 3.5.0504

  • UEFI 14.16.0017

These instructions produce the following file structure on the PXE server.

Copy
Copied!
            

/local/syslinux └── efi64 ├── ldlinux.e64 ├── libcom32.c32 ├── libutil.c32 ├── menu.c32 ├── pxelinux.0 ├── pxelinux.cfg │ └── default ├── syslinux.efi └── vesamenu.c32

  1. Install Syslinux 6.04-pre1

    1. Download and extract the syslinux archive.

      Copy
      Copied!
                  

      $ cd /local

      Copy
      Copied!
                  

      $ wget https://mirrors.edge.kernel.org/pub/linux/utils/boot/syslinux/Testing/6.04/syslinux-6.04-pre1.tar.gz

      Copy
      Copied!
                  

      $ sudo tar -xvf syslinux-6.04-pre1.tar.gz

    2. Copy the syslinux files.

      Copy
      Copied!
                  

      $ sudo ln -s /local/syslinux-6.04-pre1 /local/syslinux

      Copy
      Copied!
                  

      $ sudo mkdir /local/syslinux/efi64/pxelinux.cfg

      Copy
      Copied!
                  

      $ sudo cp syslinux-6.04-pre1/bios/core/pxelinux.0 syslinux/efi64/

      Copy
      Copied!
                  

      $ sudo cp syslinux-6.04-pre1/efi64/efi/syslinux.efi syslinux/efi64/

      Copy
      Copied!
                  

      $ sudo cp syslinux-6.04-pre1/efi64/com32/elflink/ldlinux/ldlinux.e64 syslinux/efi64/

      Copy
      Copied!
                  

      $ sudo cp syslinux-6.04-pre1/efi64/com32/menu/vesamenu.c32 syslinux/efi64/

      Copy
      Copied!
                  

      $ sudo cp syslinux-6.04-pre1/efi64/com32/menu/menu.c32 syslinux/efi64/

      Copy
      Copied!
                  

      $ sudo cp syslinux-6.04-pre1/efi64/com32/libutil/libutil.c32 syslinux/efi64

      Copy
      Copied!
                  

      $ sudo cp syslinux-6.04-pre1/efi64/com32/lib/libcom32.c32 syslinux/efi64/

  2. Edit PXE boot default configuration file as follows and make sure you update the IP address of the http server:

    Copy
    Copied!
                

    $ sudo vi /local/syslinux/efi64/pxelinux.cfg/default default menu prompt 0 timeout 300 menu title DGX Install Images label dgxbaseos-5.x.y menu label DGXBaseOS 5.x.y (Base OS) kernel http://<http-address>/dgxbaseos-5.x.y/vmlinuz initrd http://<http-address>/dgxbaseos-5.x.y/initrd append boot=live config components union=overlay noswap noeject ip::enp226s0:dhcp ethdevice-timeout=60 fetch=http://<http-address>/dgxbaseos-5.x.y/DGXOS-5.x.y-<release-labels>.iso apparmor=0 elevator=noop nvme-core.multipath=n nouveau.modeset=0 rebuild-raid offwhendone console=tty0 console=ttyS1,115200n8

The kernel boot parameters should match the contents of the corresponding ISO’s boot menu, found in /mnt/boot/grub/grub.cfg.

Live Boot Parameters

Here is some information about the live boot parameters specified in the pxelinux.cfg/default file.

Note

The man-pages for all of the live-boot parameters used above (and more) can be found here

The following parameters can be used to configure your system’s network interfaces:

ip=: Tells the initramfs to automatically configure the system’s interfaces by using DHCP. If only one interface is connected to the network, this interface should be enough. If multiple interfaces are connected to the network, then the initramfs will go with the first interface that receives a reply.

In this case, you might need to provide a more specific configuration. For instance, ip::enp1s0f0:dhcp tells the initramfs to configure the enp1s0f0 interface by using DHCP.

ethdevice-timeout: Allows you to specify how long to wait for the ethernet devices to get configured. The example above gives the system 60 seconds to configure its interfaces before giving up.

The following parameters are unique to the DGX OS Installer:

rebuild-raid: If the data RAID is specified, tells the installer to rebuild the RAID. An installation from the factory should always specify this parameter, but otherwise, it is optional.

curtin-encrypt-root: Sets up LUKS encryption on the root filesystem using a well-known passphrase.

no-mlnx-fw-update: Skips updating the NVIDIA ConnectX cards firmware during the installation.

Setup GRUB PXE Server

Instead of using syslinux to boot, you can modify the PXE server setup outlined earlier in this document to use GRUB instead. As a reminder, “the lab admin, or whoever is in charge of deploying the PXE environment, should change the directory names and structure to fit their infrastructure.” In addition, the required GRUB binaries should be downloaded and placed in the correct directories depending on the lab admin’s chosen infrastructure.

Download GRUB Packages

Create a new directory /local/grub2/ and then download the relevant packages with the correct architecture specified. In this example we will be using x86_64:

Copy
Copied!
            

$ yum install --downloadonly shim-x64.x86_64 grub2-efi-x64.x86_64

The RPMs end up in: /var/cache/yum/x86_64/7/base/packages and /var/cache/yum/x86_64/7/updates/packages. Unpack the RPMs with the following commands:

Copy
Copied!
            

$ rpm2cpio grub2-efi-x64-2.02-0.86.el7.centos.x86_64.rpm | cpio -idmv

Copy
Copied!
            

$ rpm2cpio shim-x64-15-8.el7.x86_64.rpm | cpio -idmv

Copy the following binaries from the unpacked RPMs to /local/grub2/:

  • shim.efi

  • shimx64.efi

  • grubx64.efi

    Make a copy of shimx64.efi in /local/grub2/ and rename the copy as bootx64.efi.

    Modify DHCP Configuration

    In the DHCP server’s /etc/dhcp/dhcpd.conf file:

    • Leave the “next-server” as <FTP IP_ADDR>

    • {.p dir=“ltr”} Change the “filename” entries that point to “syslinux.efi” to the following:

Copy
Copied!
            

if option architecture = 00:06 { filename "grub2/bootia32.efi"; } elsif option architecture = 00:07 { filename "grub2/bootx64.efi"; } elsif option architecture = 00:09 { filename "grub2/bootx64.efi"; } elsif option architecture = 00:0b { filename "grub2/grubaa64.efi"; } else { filename "pxelinux.0"; }

Modify TFTP Configuration

Modify /etc/xinetd.d/tftp:

  • Point the tftp-server to the grub2 directory for when the system PXE boots

  • Make the tftp-server transfer the new .efi files from the grub2 directory which are specified in the DHCP server’s dhcpd.conf file

    Once *.efi is booted it should look in the same directory for the corresponding grub.cfg file that points to vmlinuz and initrd

    Example:

    Copy
    Copied!
                

    /local/http └── http/ ├── dgxbaseos-5.x.y/ │ └── base_os_5.x.y.iso └── ftp/ ├── dgxbaseos-5.x.y/ │ └── base_os_5.x.y.iso ├── boot/ │ └── grub/ │ │ └── bios.img │ │ └── boot.cat │ │ └── grub.cfg ├── EFI/ │ └── efiboot.img ├── live/ │ └── filesystem.squashfs ├── md5sum.rtxt ├── UBUNTU_LIVE - dgxbaseos-5.x.z/ │ └── base_os_5.x.z.iso ├── boot/ │ └── grub/ │ │ └── bios.img │ │ └── boot.cat │ │ └── grub.cfg ├── EFI/ │ └── efiboot.img ├── live/ │ └── filesystem.squashfs ├── md5sum.rtxt ├── UBUNTU_LIVE - dgxbaseos-5.a.b/ ├── boot/ │ └── grub/ │ │ └── bios.img │ │ └── boot.cat │ │ └── grub.cfg ├── base_os_5.a.b.iso ├── boot/ │ └── grub/ │ │ └── bios.img │ │ └── boot.cat │ │ └── grub.cfg ├── EFI/ │ └── efiboot.img ├── live/ │ └── filesystem.squashfs ├── md5sum.rtxt ├── UBUNTU_LIVE - grub2/ ├── sdgx-5.x.y/ └── vmlinuz └── initrd ├── grub.cfg ├── grubx64.efi ├── shim.efi ├── shimx64.efi

The contents of the grub2/grub.cfg file should look something like:

Copy
Copied!
            

set default=0 set timeout=-1 insmod all_video menuentry 'dgxbaseos-5.x.y' { linux /sdgx-5.x.y/vmlinuz boot=live config components union=overlay noswap noeject ip= ethdevice-timeout=60 initrd /sdgx-5.x.y/initrd }

Optional: Configure the NVIDIA ConnextX cards to PXE boot

DGX-Servers may also PXE boot using the MLNX CX-4/5/6 cards. If you are logged into the DGX-Server host OS, and running DGX Base OS 4.4 or later, then you can perform this section’s steps using the /usr/sbin/mlnx_pxe_setup.bash tool, which will enable the UEFI PXE ROM of every MLNX Infiniband device found.

Otherwise, proceed with the manual steps below.

Query the UEFI PXE ROM State

Here are the steps to query the current state of the UEFI PXE ROM. To PXE boot from the MLNX CX-4/5/6 cards, enable the UEFI PXE ROM of the card from which you want to PXE boot.

By default, the UEFI PXE ROM is disabled.

Note

This process cannot be completed done remotely, because it needs to be performed from the host OS.

The configuration of the MLNX CX devices is handled through the [mlxconfig] command.

To determine the device name and current configurations of the MLNX CX cards, run the [sudo mlxconfig query]

Copy
Copied!
            

$ sudo mlxconfig query Device #1: ---------- Device type: ConnectX4 Name: MCX455A-ECA_Ax Description: ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6 Device: /dev/mst/mt4115_pciconf3 Configurations: Next Boot ... ... EXP_ROM_UEFI_x86_ENABLE False(0) ... ...

Enable the UEFI PXE ROM

Here is some information about how to enable the UEFI PXE ROM.

  1. Ensure that the EXP_ROM_UEFI_x86_ENABLE configuration is set to True(1) for the MLNX CX card from which you want to PXE boot.

    Copy
    Copied!
                

    $ sudo mlxconfig -y -d /dev/mst/mt4115_pciconf3 set EXP_ROM_UEFI_x86_ENABLE=1

  2. Reboot the system:

    Copy
    Copied!
                

    $ sudo reboot


  3. Wait for the system to reboot and confirm that the configuration was set.

    Copy
    Copied!
                

    $ sudo mlxconfig query Device #1: ---------- Device type: ConnectX4 Name: MCX455A-ECA_Ax Description: ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6 Device: /dev/mst/mt4115_pciconf3 Configurations: Next Boot ... ... EXP_ROM_UEFI_x86_ENABLE True(1) ... ...

Curtin Customizations

Here is some information about customizing the curtin files.

For some deployments, you may want to use your own curtin files. This section will describe some sections contained in the built-in curtin files as well as how to perform some common customizations.

Customizing the Curtin YAML

The curtin file is located in the filesystem.squashfs. Creating a touchless install, such as by seeding the initial userid and password, and skipping the oem-config steps, is accomplished by customizing the curtin file. (nooemconfig boot parameter described above does initial userid/password seeding and oem-config skip. For any other customization to curtin yaml, follow the steps below).

  1. Get the filesystem.squashfs from the ISO

  2. Run “unsquashfs filesystem.squashfs”

  3. Copy the curtin file for the platform you’re trying to PXE install from the squashfs-root/curtin/ directory into a webserver or FTP server.

  4. Customize the curtin file

  5. Use the force-curtin parameter in your PXE bootloader entry to point to the URL of your new curtin file

More info about curtin can be found here: https://curtin.readthedocs.io/en/latest/index.html

An example curtin file for performing touchless installs can be found here: https://gitlab-master.nvidia.com/jamien/jamien-work/-/blob/master/curtin/fast-curtin.yaml

NVIDIA-Specific Curtin Variables

Here is some information about NVIDIA-specific curtin variables.

The curtin files contained in the ISO are platform-specific, and serve as a good starting point for custom versions. Many of them contain variables, prefixed with CHANGE_, that get substituted by the installer.

CHANGE_STORAGE_REG

This gets removed and uncommented when the boot parameter curtin-encrypt-root is not present. Uncommenting this stanza results in the standard disk partitioning scheme without LUKS encryption.

CHANGE_STORAGE_ENC

This gets removed and uncommented when the boot parameter “curtin-encrypt-root” is present. Uncommenting this stanza results in an encrypted root partition.

CHANGE_BOOT_DISK_NAME_x

This is a disk-name, without the “/dev” prefix. There may be multiple ones (i.e. CHANGE_BOOT_DISK_NAME_1 and CHANGE_BOOT_DISK_NAME_2) for platforms that expect a RAIDed boot device as is the case for DGX-2 and DGX A100. The installer will find the appropriate disk name to substitute here. Alternatively, the “force-bootdisk” parameter can be used to specify the disk name(s).

CHANGE_BOOT_DISK_PATH_x: This is the same as the CHANGE_BOOT_DISK_NAME_x variable above, except that is prefixed with /dev/.

CHANGE_POWER_STATE: This tells the installer whether to halt or reboot the machine after the install is complete. It is controlled by the “offwhendone” boot parameter.

CHANGE_DESC_PLATFORM: The installer will substitute this with a platform-specific descriptive name.

CHANGE_SERIAL_NUMBER: The installer will substitute this with the serial number reported by dmidecode

CHANGE_MLNX_FWUPD: This value either gets removed or substituted with “RUN_FW_UPDATER=no” by the installer. This is done based on whether or not the “no-mlnx-fw-update” boot parameter is present. This determines whether or not the installer will update the FW on Mellanox networking cards that are present on the machine.

CHANGE_INSTALL_PKGS: The installer will substitute this value with a list of packages specific to the platform. The lists of packages are specified by the *-pkgs files in the squashfs.

CHANGE_REBUILD_RAID: This gets replaced with either “true” or “false” based on whether or not the “rebuild-raid” boot parameter is present.

CHANGE_IPMISOL: This gets replaced with either “true” or “false” based on whether or not the “curtin-encrypt-root” boot parameter is present. When we set the system up with encryption, we also undo the IPMI serial-over-LAN configuration to ensure that the LUKS passphrase prompt shows up on the console rather than the serial-over-LAN interface.

While it is possible to replace these values on your own, we strongly recommend letting the installer handle this.

Boot the DGX System over PXE

You can power cycle the DGX A100 through BMC GUI, or, alternatively, use “ipmitool” to set pxe boot and reboot the client DGX A100:

Copy
Copied!
            

$ ipmitool -I lanplus -H <DGX-BMC-IP> -U <username> -P <password> chassis bootdev pxe options=efiboot

Copy
Copied!
            

$ ipmitool -I lanplus -H <DGX-BMC-IP> -U <username> -P <password> chassis power reset

Troubleshooting

During the PXE boot process, you may see following errors from client BMC:

Copy
Copied!
            

BootCurrent: 0018 Timeout: 1 seconds BootOrder: 0000,0006,0007,0008,000C,000D,000E,000F,0016,0017,0001 Boot0000* ubuntu Boot0001 UEFI: Built-in EFI Shell Boot0004* UEFI: PXE IPv4 Intel(R) I210 Gigabit Network Connection Boot0005* UEFI: PXE IPv6 Intel(R) I210 Gigabit Network Connection Boot0006* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:5F:44:7C Boot0007* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:74:74:9E Boot0008* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:74:74:8E Boot0009* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:74:6B:CA Boot000A* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:74:74:9A Boot000B* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:5F:51:D8 Boot000C* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:74:6A:FA Boot000D* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:74:74:C2 Boot000E* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:4A:62:60 Boot000F* UEFI: PXE IPv4 Mellanox Network Adapter - 0C:42:A1:4A:62:61 Boot0016* UEFI: PXE IPv6 Mellanox Network Adapter - 0C:42:A1:4A:62:60 Boot0017* UEFI: PXE IPv6 Mellanox Network Adapter - 0C:42:A1:4A:62:61 Invalid BootOrder order entry value0018 ^ efibootmgr: entry 0018 does not exist Unexpected error while running command. Command: ['unshare', '--fork', '--pid', '--', 'chroot', '/tmp/tmpsgpxpe9k/target', 'efibootmgr', '-o', '0018,0000,0006,0007,0008,000C,000D,000E,000F,0016,0017,0001'] Exit code: 8

The solution is to wait for the Ubuntu console to appear, login as root without password, then run the following commands to make the Intel I210 NIC as the first NIC to try PXE boot:

Copy
Copied!
            

$ efibootmgr -o 0004,0000,0006,0007,0008,0009,000A,000B, 000C,000D,000E,000F

Reference: Other IPMI Boot Options

Here is some information about other IPMI boot options.

© Copyright 2020-2023, NVIDIA. Last updated on Mar 24, 2023.