NVIDIA DOCA Release Notes

Host

DOCA SDK

2.5.0

Software development kit package for developing host software

DOCA Runtime

2.5.0

Runtime libraries required to run DOCA-based software applications on host

DOCA Tools

2.5.0

Tools for developers and administrators on host

DOCA Extra

2.5.0

Contains helper scripts (doca-info, doca-kernel-support)

DOCA OFED

2.5.0

Software stack which operates across all NVIDIA network adapter solutions

Arm emulated (QEMU) development container

4.5.0

Linux-based BlueField Arm emulated container for developers

Target BlueField DPU (Arm)

BlueField BSP

4.5.0

BlueField image and firmware

DOCA SDK

2.5.0

Software development kit packages for developing Arm software

DOCA Runtime

2.5.0

Runtime libraries required to run DOCA-based software applications on Arm

DOCA Tools

2.5.0

Tools for developers and administrators for Arm target

3631257

Description: When installing DOCA for host on a SLES15.5 system that uses NVMe storage (e.g., NVMe disk) and a non-default kernel that requires rebuilding the kernel, the script doca-kernel-support builds and installs the mlnx-nvme drivers that override standard NVMe kernel drivers, which renders the system unbootable.

Workaround: After installing doca-extra, edit the doca-kernel-extra script before running it to avoid it building mlnx-nvme modules. The option --without-mlnx-nvme must be added to the invocation of install.pl:

            

            
sed -i -e '/install\.pl/a\\t--without-mlnx-nvme \\' /opt/mellanox/doca/tools/doca-kernel-support
        

Keyword: NVMe; SLES

Reported in version: 2.5.0

3678069

Description: If using DPUs with NVMe and mmcbld and configured to boot from mmcblk, users must create bf.cfg file with device=/dev/mmcblk0, then install the *.bfb as normal.

Workaround: N/A

Keyword: NVMe

Reported in version: 2.5.0

3669736

Description: DOCA can run only one instance of a DOCA library per process on setups with a page size different than 4K (e.g., Rockey). Combination of different libraries cannot be run on the same process.

Workaround: N/A

Keyword: OpenEuler; Rocky; DOCA libs

Reported in version: 2.5.0

3469692

Description: The overall number of IPsec sessions is limited to 64K per port.

Workaround: N/A

Keyword: IPsec

Reported in version: 2.5.0

3680538

Description: When using strongSwan or OVS-IPsec as explained in the NVIDIA BlueField DPU BSP, the IPSec Rx data path is not offloaded to hardware and occurs in software running on the Arm cores. As a result, bandwidth performance is substantially low.

Workaround: N/A

Keyword: IPsec

Reported in version: 2.5.0

3674295

Description: The script for the East-West Overlay Encryption application is not supported in DOCA 2.5.0. Users who wish to configure OVS IPsec with hardware offload must should follow the steps explained in the "Configuration Flow" section of the guide instead of running the application.

Workaround: N/A

Keyword: IPsec

Reported in version: 2.5.0

N/A

Description: Execution unit partitions are still not implemented and would be added in future releases.

Workaround: N/A

Keyword: EU tool

Reported in version: 2.5.0

3618936

Description: When moving to DPU mode from NIC mode, it is necessary to reinstall the BFB and perform a graceful reboot to the DPU by shutting down the Arm cores before rebooting the host system.

Workaround: N/A

Keyword: NIC mode

Reported in version: 2.5.0

3666160

Description: Installing BFB using bfb-install when mlxconfig PF_TOTAL_SF>1700, triggers server reboot immediately.

Workaround: Change PF_TOTAL_SF to 0, perform a graceful shutdown, power cycle, then installing BFB.

Keyword: SF; PF_TOTAL_SF; BFB installation

Reported in version: 2.2.1

3603146

Description: Running mlxfwreset on BlueField-3 may cause the external host to crash when the RShim driver is running on that host.

Workaround: Stop the RShim driver on the external host using systemctl stop rshim before performing mlxfwreset .

Keyword: RShim; mlxfwreset

Reported in version: 2.2.1

3594836

Description: When enabling Flex IO SDK tracer at high rates, a slow-down in processing may occur and/or some traces may be lost.

Workaround: Keep tracing limited to ~1M traces per second to avoid a significant processing slow-down. Use tracer for debug purposes and consider disabling it by default.

Keyword: Tracer FlexIO

Reported in version: 2.2.1

3592080

Description: When using UEK8 on the host in DPU mode, creating a VF on the host consumes about 100MB memory on the DPU.

Workaround: N/A

Keyword: UEK; VF

Reported in version: 2.2.1

3568924

Description: doca_rdma does not support cross-subnet routing.

Workaround: N/A

Keyword: RDMA

Reported in version: 2.2.1

3566042

Description: Virtio hotplug is not supported in GPU-HOST mode on the NVIDIA Converged Accelerator.

Workaround: N/A

Keyword: Virtio; Converged Accelerator

Reported in version: 2.2.0

3546474

Description: PXE boot over ConnectX interface might not work due to an invalid MAC address in the UEFI boot entry.

Workaround: On the DPU, create /etc/bf.cfg file with the relevant PXE boot entries, then run the command bfcfg.

Keyword: PXE; boot; MAC

Reported in version: 2.2.0

3561723

Description: Running mlxfwreset sync 1 on NVIDIA Converged Accelerators may be reported as supported although it is not. Executing the reset will fail.

Workaround: N/A

Keywords: mlxfwreset

Reported in version: 2.2.0

3546202

Description: After rebooting a BlueField-3 DPU running Rocky Linux 8.6 BFB, the kernel log shows the following error:

            

            
[    3.787135] mlxbf_gige MLNXBF17:00: Error getting PHY irq. Use polling instead
        

This message indicates that the Ethernet driver will function normally in all aspects, except that PHY polling is enabled.

Workaround: N/A

Keywords: Linux; PHY; kernel

Reported in version: 2.2.0

3306489

Description: When performing longevity tests (e.g., mlxfwreset, DPU reboot, burning of new BFBs), a host running an Intel CPU may observer errors related to "CPU 0: Machine Check Exception".

Workaround: Add intel_idle.max_cstate=1 entry to the kernel command line.

Keywords: Longevity; mlxfwreset; DPU reboot

Reported in version: 2.2.0

3529297

Description: Enhanced NIC mode is not supported on BlueField-2 DPUs.

Workaround: N/A

Keywords: Operation; mode

Reported in version: 2.2.0

3538486

Description: When removing LAG configuration from the DPU, a kernel warning for uverbs_destroy_ufile_hw is observed if virtio-net-controller is still running.

Workaround: Stop virtio-net-controller service before cleaning up bond configuration.

Keywords: Virtio-net; LAG

Reported in version: 2.2.0

3534219

Description: On BlueField-3 devices, from DOCA 2.2.0 to 32.37.1306 (or lower), the host crashes when executing partial Arm reset (e.g., Arm reboot; BFB push; mlxfwreset).

Workaround: Before downgrading the firmware:

1. Run:

   Copy

   Copied!

               
   
               
   echo 0 > /sys/bus/platform/drivers/mlxbf-bootctl/large_icm
           

2. Reboot Arm.

Keyword: BlueField-3; downgrade

Reported in version: 2.2.0

3516279

Description: The NVIDIA DOCA East-West Overlay Encryption Reference Application (and the underlying DPU OS Kernel driver IPsec functionality) is not supported. User space DOCA IPsec is not impacted.

Workaround: N/A

Keyword: IPsec

Reported in version: 2.2.0

3444073

Description: mlxfwreset is not supported in this release.

Workaround: Perform a graceful shutdown, then power cycle the host.

Keyword: mlxfwreset; support

Reported in version: 2.0.2

3462630

When trying to perform a PXE installation when UEFI Secure Boot is enabled, the following error messages may be observed:

            

            
error: shim_lock protocol not found.
error: you need to load the kernel first.
        

Workaround: Download a Grub EFI binary from the Ubuntu website. For further information on Ubuntu UEFI Secure Boot PXE Boot, please visit Ubuntu's official website.

Keyword: PXE; UEFI Secure Boot

Reported in version: 2.0.2

3448841

Description: While running CentOS 8.2, switchdev Ethernet DPU runs in "shared" RDMA net namespace mode instead of "exclusive".

Workaround: Use ib_core module parameter netns_mode=0. For example:

            

            
echo "options ib_core netns_mode=0" >> /etc/modprobe.d/mlnx-bf.conf
        

Keyword: RDMA; isolation; Net NS

Reported in version: 2.0.2

3232444

Description: After live migration of virtio-net devices using the VFE driver, the max_queues_size output from the virtnet list may be wrong. This does not affect the actual value.

Workaround: N/A

Keyword: Virtio-net; live migration

Reported in version: 2.0.2

3441287

Description: Failure occurs when attempting to raise static LAG with ifenslave_2.10ubuntu3 package.

Workaround: Use ifenslave_2.9ubuntu1.

Keywords: ifenslave; bonding

Reported in version: 4.0.2

2706803

Description: When an NVMe controller, SoC management controller, and DMA controller are configured, the maximum number of VFs is limited to 124.

Workaround: N/A

Keyword: VF; limitation

Reported in version: 2.0.2

3273435

Description: Changing the mode of operation between NIC and DPU modes results in different capabilities for the host driver which might cause unexpected behavior.

Workaround: Reload the host driver or reboot the host.

Keyword: Modes of operation; driver

Reported in version: 2.0.2

3264749

Description: In Rocky and CentOS 8.2 inbox-kernel BFBs, RegEx requires the following extra huge page configuration for it to function properly:

            

            
sudo hugeadm --pool-pages-min DEFAULT:2048M
sudo systemctl start mlx-regex.service
systemctl status mlx-regex.service
        

If these commands have executed successfully you should see active (running) in the last line of the output.

Workaround: N/A

Keyword: RegEx; hugepages

Reported in version: 1.5.1

3240153

Description: DOCA kernel support only works on a non-default kernel.

Workaround: N/A

Keyword: Kernel

Reported in version: 1.5.0

3217627

Description: The doca_devinfo_rep_list_create API returns success on the host instead of Operation not supported.

Workaround: N/A

Keyword: DOCA core; InfiniBand

Reported in version: 1.5.0