NVIDIA DOCA Release Notes
NVIDIA DOCA SDK release notes containing information on new features, software interoperability, and known issues.
DOCA 2.7.0 introduces NVIDIA® BlueField® networking platform enhancement for high-performance and secure AI bare-metal cloud and DOCA-Host updates for supported BlueField and NVIDIA® ConnectX® devices. With programmable congestion control (PCC) and data-path acceleration (DPA). DOCA SDK provides an extensive framework for developers.
BlueField-3 devices are not supported with MLNX_OFED as the host driver and are required to use DOCA-Host.
Spectrum-X 1.0.1 with BlueField-3 SuperNIC
DOCA PCC (GA) – Added new telemetry information to the PCC application
DOCA Flow enhancements, including DOCA Flow Tune Server and Pipeline Visualization for debugging (alpha support)
Added support for DOCA Flow switch unified model
NoteDOCA Flow switch has now unified all the representor ports to the switch manager port for traffic management. User should only manage the pipes on the switch manager port.
Applications must not call DPDK rte_eth_dev_start/configure/stop for VF/SF representors anymore. User should acquire packets (e.g., with DPDK but calling rte_eth_rx/tx_burst()) only with switch manager port.
OVS-DOCA – Unified representor for multiple ports for better resource utilization with higher scale; OVS package rename for smoother installation
Increased support for virtio-net VF devices on BlueField-3 networking platforms to 2K
DOCA HBN Service 2.2.0 enhancements, including GA-level support for Local VRF Route Leaking, EVPN Downstream VNI (DVNI) for symmetric EVPN Route Leaking, Network-to-Network Hairpin routing support on BlueField uplinks
SNAP Encryption at Rest with Zero-Copy: Available with BlueField-3 with SNAP 4.4.0
DOCA Firefly Service enhancements, including new Telco profile, ptp4l update, new Firefly servo module
Traffic Crypto - DOCA IPsec GA and merge into DOCA Flow, New Security protocol - PSP
Alpha support for new DOCA Unified Resource and Offload Management (UROM) library
Alpha support for DOCA Device Emulation (DevEmu) library – Emulate your own standard/non-standard devices on BlueField
DOCA GPUNetIO new API to support RDMA
DOCA Comm Channel (Comch) API update, Extend Comch to Arm↔DPA, host↔DPA.
NoteComch API is being updated, the old version is in deprecation process, DOCA 2.8 will be the last version to support it.
DOCA Remote Direct Memory Operation (RDMO) reference application
Alpha support for DOCA Management Service (DMS) – simplifying BlueField post-boot provisioning and configuration using standard configuration interfaces (API/CLI)
DOCA NVQual – H20-BFx support, power stressors improvements
DOCA NVCert – BlueField-3 SuperNIC support, SPC-X support using multi-DPU (and ConnectX) and GPU direct, RDMA/TCP-OVS/IPSec and VXLAN workloads
Updated the default operation mode of SuperNICs to NIC mode (from DPU mode). This is relevant to the following SKUs:
900-9D3B4-00CC-EA0
900-9D3B4-00SC-EA0
900-9D3B4-00CV-EA0
900-9D3B4-00SV-EA0
900-9D3B4-00EN-EA0
900-9D3B4-00PN-EA0
900-9D3D4-00EN-HA0
900-9D3D4-00NN-HA0
NoteWhen upgrading one of these SuperNICs to 2.7.0, if its mode of operation was changed at any point in the past, then the last configured mode of operation will remain unchanged. Otherwise, the SuperNIC will rise in NIC operation mode.
DOCA packaging – new BlueField firmware bundle package (bf-fwbundle-<version>.prod.bfb), a smaller image for Day 2 upgrades, without the OS and DOCA runtime. Includes ATF, UEFI, nic-fw, bmc-fw, and eROT only.
BlueField-3 Firmware Components Upgrade – Upgrade all BlueField-3 firmware components in one upgrade flow through either bfb-install from the host (via RShim), or DPU BMC Redfish transfer BFB image
Update BlueField NIC-Firmware automatically as part of .bfb image upgrade
Improved BlueField BMC robustness –
Report LLDP for L2 discovery via Redfish
Improved BlueField DPU debuggability
Compilation on top of DOCA's SDK
DOCA 2.7 – installation now includes additional pkg-config (.pc) definitions per DOCA SDK library on top of the general doca.pc file. This is part of a deprecation process for doca.pc and a focus on modularity of DOCA's SDK.
Please refer to DOCA SDK reference samples and applications for an example of using the per library .pc files.
NoteStarting with DOCA 2.8, the general doca.pc file will be removed from the release and only files per DOCA SDK library will remain.
Added support for new BlueField reset and reboot procedures for loading new firmware and firmware configuration changes which replace previous need for server power cycle
The format of image filenames for the BF-Bundle and DOCA-Host have been updated to the following template:
BF-Bundle image file format – bf-bundle-<doca_ver.LTS#>-<build#>[BUILD-LABEL]-<yy.mm>-<OS_distro>-<#os_ver>[OS-LABEL]-<unsigned/dev/prod>.<bfb/iso>
DOCA-Host image file format – doca-host-<doca_ver.LTS#>-<build#>[BUILD-LABEL]-<yy.mm>-<OS_distro>-<#os_ver>[OS-LABEL]-<arch>.<rpm/deb/iso>
Where:
<doca_ver.LTS#>-<build#> – the DOCA version with the NVIDIA build number in a x.y.z-abcd format (e.g., 2.7.0-1456). If it is an LTS release, it indicates which update number it is.
<yy.mm> – the year and month the image was created/released (e.g., 24.04)
<OS_distro>-<#os_ver> – the name and version of the operating system (e.g., ubuntu-22.04)
<arch>.<rpm/deb/iso> – which processor architecture is supported and how the image is packaged (e.g., x86.rpm or x86.deb)
<unsigned/dev/prod>.<bfb/iso> – security signature of the image (no signature, development, production) and how the image is packaged (e.g., prod.rpm or dev.rpm)
Refer to the NVIDIA DOCA Installation Guide for Linux for information on:
Setting up DOCA SDK on your BlueField networking platform or SmartNIC
Supported BlueField platforms
By default, installing DOCA profiles with standard Linux tools (yum, apt) installs both doca-runtime and doca-devel (previously doca-sdk).
doca-runtime includes all the components, libs, drivers, and tools used in the production environment by the DOCA admin
doca-devel includes all the components, libs, drivers, and tools used for development, including reference applications, compilers, etc.
Starting with DOCA 2.8, the default installation of BlueField-Bundle and DOCA-Host profiles will only include DOCA runtime. doca-devel can be installed manually as needed.
Embedded DOCA Libraries
Component |
Version |
doca-apps |
2.7.0 |
doca-grpc |
2.7.0 |
doca-libs |
2.7.0 |
ucx |
1.17.0-1.2404066 |
gpunetio |
2.7.0 |
Embedded DOCA Firmware Components
Component |
Version |
Description |
ATF |
Arm-trusted firmware is a reference implementation of secure world software for Arm architectures |
|
UEFI |
UEFI is a specification that defines the architecture of the platform firmware used for booting and its interface for interaction with the operating system |
|
BlueField-3 NIC firmware |
Firmware is used to run user programs on the BlueField-3 which allow hardware to run |
|
BlueField-2 NIC firmware |
Firmware is used to run user programs on the BlueField-2 which allow hardware to run |
|
BMC firmware |
BlueField BMC firmware |
|
BlueField-3 eROT (Glacier) |
cec_ota_BMGP-04.0f |
BlueField-3 eROT firmware |
BlueField-2 eROT (CEC) |
00.02.0182.0000.n02 |
BlueField-2 eROT firmware |
Embedded DOCA Drivers
Component |
Version |
Description |
collectx-clxapi |
1.17.0-1 |
A library which exposes the CollectX API, which allows any 3rd party to easily use CollectX functionality in their own programs |
doca-base (MLNX_OFED) |
NVIDIA® MLNX_OFED is a single software stack that operates across all NVIDIA network adapter solutions |
|
dpacc |
1.7.0-1 |
DPACC is a high-level compiler for the DPA processor which compiles code targeted for the data-path accelerator (DPA) processor into a device executable and generates a DPA program |
dpcp |
1.1.48-1.2404066 |
DPCP provides a unified flexible interface for programming IB devices using DevX |
flexio |
24.04.2148-0 |
FlexIO SDK exposes an API for managing the device and executing native code over the DPA processor |
ibutils |
2.1.1 |
ibdiagnet scans the fabric using directed route packets and extracts all the available information regarding its connectivity and devices. |
libvma |
9.8.60-1 |
The NVIDIA® VMA library accelerates latency-sensitive and throughput-demanding TCP and UDP socket-based applications by offloading traffic from the user-space directly to the NIC, without going through the kernel and the standard IP stack (kernel-bypass) |
libxlio |
3.30.5-1.2404066 |
The NVIDIA® XLIO software library boosts the performance of TCP/IP applications based on NGINX (CDN, DoH, etc.) and storage solutions as part of the SPDK |
MFT |
NVIDIA® MFT is a set of firmware management and debug tools for NVIDIA devices |
|
mlnx-dpdk |
22.11.0-2404 |
Equivalent to DPDK upstream. The versioning of MLNX_DPDK indicates which upstream DPDK it is compatible with it (e.g., 22.11 is compatible with upstream DPDK 2022.11). |
mlnx-libsnap |
1.6.0-1 |
Libsnap is a common library designed to assist common tasks for applications wishing to interact with emulated hardware over BlueField and take the most advantage from hardware capabilities |
mlnx-snap |
3.8.0-3 |
BlueField SNAP for NVMe and virtio-blk enables hardware-accelerated virtualization of local storage |
mlx-regex |
1.2-ubuntu1 |
RegEx is a library that provides RegEx pattern matching to DOCA applications using the regular expression processor (RXP) or software-based engines when required |
OpenSM |
InfiniBand Subnet Manager and Subnet Administrator based on OpenSM |
|
Rivermax |
1.50.7 |
NVIDIA® Rivermax® is an optimized networking SDK for media and data streaming applications |
RShim |
2.0.27 |
The user-space driver to access the BlueField SoC via the RShim interface, providing ways to push boot stream, debug the target, or login via the virtual console or network interface |
SHARP |
Improves the performance of MPI and Machine Learning collective operation by offloading from CPUs and GPUs to the network and eliminating the need to send data multiple times between endpoints |
|
SPDK |
23.01.5-20 |
SPDK provides a set of tools and libraries for writing high performance, scalable, user-mode storage applications |
Virtio-net-controller |
1.9.17-1 |
Virtio-net-controller is a systemd service running on BlueField, with a user interface front-end to communicate with the background service |
VMA |
Accelerates latency-sensitive and throughput-demanding TCP and UDP socket-based applications by offloading traffic from the user-space directly to the network interface card (NIC) or Host Channel Adapter (HCA) |
|
XLIO |
Boosts the performance of TCP/IP applications based on NGINX (CDN, DoH, etc.) and storage solutions as part of the SPDK |
DOCA Packages
Device |
Component |
Version |
Description |
Host |
DOCA Devel |
2.7.0 |
Software development kit package and tools for developing host software |
DOCA Runtime |
2.7.0 |
Runtime libraries and tools required to run DOCA-based software applications on host |
|
DOCA Extra |
2.7.0 |
Contains helper scripts (doca-info, doca-kernel-support) |
|
DOCA OFED |
2.7.0 |
Software stack which operates across all NVIDIA network adapter solutions |
|
Arm emulated (QEMU) development container |
4.7.0 |
Linux-based BlueField Arm emulated container for developers |
|
Target BlueField DPU (Arm) |
BlueField BSP |
4.7.0 |
BlueField image and firmware |
DOCA SDK |
2.7.0 |
Software development kit packages and tools for developing Arm software |
|
DOCA Runtime |
2.7.0 |
Runtime libraries and tools required to run DOCA-based software applications on Arm |
Supported Host OS per DOCA-Host Installation Profile
Starting with DOCA version 2.6.0 OSs with kernel versions lower than 4.18 will no longer be supported. DOCA 2.5.0 is the last version to support OS with kernel lower than 4.18.
The default operating system included with the BlueField Bundle (for DPU and SuperNIC) is Ubuntu 22.04.
The supported operating systems on the host machine per DOCA-Host installation profile are the following:
Only the following generic kernel versions are supported for DOCA local repo package for host installation.
OS |
OS Version |
Default Kernel Version |
Arch |
DOCA Profile |
||
doca-all |
doca-networking |
doca-ofed |
||||
Alinux |
3.2 |
5.10.134-13.al8.x86_64 |
x86 |
✔ |
✔ |
✔ |
Anolis |
8.6 |
5.10.134+ |
aarch64 |
✔ |
||
x86 |
✔ |
|||||
BCLinux |
21.10SP2 |
4.19.90-2107.6.0.0098.oe1.bclinux.aarch64 |
aarch64 |
✔ |
||
4.19.90-2107.6.0.0100.oe1.bclinux.x86_64 |
x86 |
✔ |
||||
CTYunOS |
2.0 |
4.19.90-2102.2.0.0062.ctl2.aarch64 |
aarch64 |
✔ |
||
4.19.90-2102.2.0.0062.ctl2.x86_64 |
x86 |
✔ |
||||
3.0 (23.01) |
5.10.0-136.12.0.86.ctl3.aarch64 |
aarch64 |
✔ |
✔ |
✔ |
|
5.10.0-136.12.0.86.ctl3.x86_64 |
x86 |
✔ |
✔ |
✔ |
||
Debian |
10.13 |
4.19.0-21-arm64 |
aarch64 |
✔ |
||
4.19.0-21-amd64 |
x86 |
✔ |
||||
10.8 |
4.19.0-14-arm64 |
aarch64 |
✔ |
|||
4.19.0-14-amd64 |
x86 |
✔ |
✔ |
✔ |
||
10.9 |
4.19.0-16-amd64 |
x86 |
✔ |
|||
11.3 |
5.10.0-13-arm64 |
aarch64 |
✔ |
|||
5.10.0-13-amd64 |
x86 |
✔ |
||||
12.1 |
6.1.0-10-arm64 |
aarch64 |
✔ |
|||
6.1.0-10-amd64 |
x86 |
✔ |
||||
EulerOS |
2.0sp11 |
5.10.0-60.18.0.50.h323.eulerosv2r11.aarch64 |
aarch64 |
✔ |
||
5.10.0-60.18.0.50.h323.eulerosv2r11.x86_64 |
x86 |
✔ |
||||
2.0sp12 |
5.10.0-136.12.0.86.h1032.eulerosv2r12.aarch64 |
aarch64 |
✔ |
|||
5.10.0-136.12.0.86.h1032.eulerosv2r12.x86_64 |
x86 |
✔ |
||||
Kylin |
10sp2 |
4.19.90-24.4.v2101.ky10.aarch64 |
aarch64 |
✔ |
||
4.19.90-24.4.v2101.ky10.x86_64 |
x86 |
✔ |
||||
10sp3 |
4.19.90-52.22.v2207.ky10.aarch64 |
aarch64 |
✔ |
|||
4.19.90-52.22.v2207.ky10.x86_64 |
x86 |
✔ |
||||
Mariner |
2.0 |
5.15.118.1-1.cm2.x86_64 |
x86 |
✔ |
||
Oracle Linux |
7.9 |
5.4.17-2011.6.2.el7uek.x86_64 |
x86 |
✔ |
||
8.4 |
5.4.17-2102.201.3.el8uek.x86_64 |
x86 |
✔ |
|||
8.6 |
5.4.17-2136.307.3.1.el8uek.x86_64 |
x86 |
✔ |
|||
8.7 |
5.15.0-3.60.5.1.el8uek.x86_64 |
x86 |
✔ |
✔ |
✔ |
|
8.8 |
5.15.0-101.103.2.1.el8uek.x86_64 |
x86 |
✔ |
|||
9.1 |
5.15.0-3.60.5.1.el9uek.x86_64 |
x86 |
✔ |
|||
9.2 |
5.15.0-101.103.2.1.el9uek.x86_64 |
x86 |
✔ |
|||
openEuler |
20.03sp3 |
4.19.90-2112.8.0.0131.oe1.aarch64 |
aarch64 |
✔ |
||
4.19.90-2112.8.0.0131.oe1.x86_64 |
x86 |
✔ |
||||
22.03 |
5.10.0-60.18.0.50.oe2203.aarch64 |
aarch64 |
✔ |
|||
5.10.0-60.18.0.50.oe2203.x86_64 |
x86 |
✔ |
||||
RHEL/CentOS |
8.0 |
4.18.0-80.el8.aarch64 |
aarch64 |
✔ |
||
4.18.0-80.el8.x86_64 |
x86 |
✔ |
||||
8.1 |
4.18.0-147.el8.aarch64 |
aarch64 |
✔ |
|||
4.18.0-147.el8.x86_64 |
x86 |
✔ |
||||
8.2 |
4.18.0-193.el8.aarch64 |
aarch64 |
✔ |
|||
4.18.0-193.el8.x86_64 |
x86 |
✔ |
✔ |
✔ |
||
8.3 |
4.18.0-240.el8.aarch64 |
aarch64 |
✔ |
|||
4.18.0-240.el8.x86_64 |
x86 |
✔ |
||||
8.4 |
4.18.0-305.el8.aarch64 |
aarch64 |
✔ |
|||
4.18.0-305.el8.x86_64 |
x86 |
✔ |
||||
RHEL/Rocky |
8.5 |
4.18.0-348.el8.aarch64 |
aarch64 |
✔ |
||
4.18.0-348.el8.x86_64 |
x86 |
✔ |
||||
8.6 |
4.18.0-372.41.1.el8_6.aarch64 |
aarch64 |
✔ |
|||
4.18.0-372.41.1.el8_6.x86_64 |
x86 |
✔ |
✔ |
✔ |
||
8.7 |
4.18.0-425.14.1.el8_7.aarch64 |
aarch64 |
✔ |
|||
4.18.0-425.14.1.el8_7.x86_64 |
x86 |
✔ |
||||
8.8 |
4.18.0-477.10.1.el8_8.aarch64 |
aarch64 |
✔ |
✔ |
✔ |
|
4.18.0-477.10.1.el8_8.x86_64 |
x86 |
✔ |
✔ |
✔ |
||
8.9 |
4.18.0-513.5.1.el8_9.aarch64 |
aarch64 |
✔ |
✔ |
✔ |
|
4.18.0-513.5.1.el8_9.x86_64 |
x86 |
✔ |
✔ |
✔ |
||
9.0 |
5.14.0-70.46.1.el9_0.aarch64 |
aarch64 |
✔ |
|||
5.14.0-70.46.1.el9_0.x86_64 |
x86 |
✔ |
||||
9.1 |
5.14.0-162.19.1.el9_1.aarch64 |
aarch64 |
✔ |
|||
5.14.0-162.19.1.el9_1.x86_64 |
x86 |
✔ |
✔ |
✔ |
||
9.2 |
5.14.0-284.11.1.el9_2.aarch64 |
aarch64 |
✔ |
|||
5.14.0-284.11.1.el9_2.x86_64 |
x86 |
✔ |
||||
9.3 |
5.14.0-362.8.1.el9_3.aarch64 |
aarch64 |
✔ |
|||
5.14.0-362.8.1.el9_3.x86_64 |
x86 |
✔ |
||||
SLES |
15sp2 |
5.3.18-22-default |
aarch64 |
✔ |
||
x86 |
✔ |
|||||
15sp3 |
5.3.18-57-default |
aarch64 |
✔ |
|||
x86 |
✔ |
|||||
15sp4 |
5.14.21-150400.22-default |
aarch64 |
✔ |
|||
x86 |
✔ |
|||||
15sp5 |
5.14.21-150500.53-default |
aarch64 |
✔ |
|||
x86 |
✔ |
|||||
TKLinux |
3.3 |
5.4.119-19.0009.39 |
aarch64 |
✔ |
||
5.4.119-19.0009.39 |
x86 |
✔ |
||||
Ubuntu |
20.04 |
5.4.0-26-generic |
aarch64 |
✔ |
||
x86 |
✔ |
✔ |
✔ |
|||
22.04 |
5.15.0-25-generic |
aarch64 |
✔ |
✔ |
✔ |
|
x86 |
✔ |
✔ |
✔ |
|||
24.04 |
6.8.0-31-generic |
aarch64 |
✔ |
|||
x86 |
✔ |
|||||
UOS |
20.1060a |
5.10.0-46.uelc20.aarch64 |
aarch64 |
✔ |
||
5.10.0-46.uelc20.x86_64 |
x86 |
✔ |
||||
20.1060e |
5.10.0-46.uel20.aarch64 |
aarch64 |
✔ |
|||
5.10.0-46.uel20.x86_64 |
x86 |
✔ |
||||
XenServer |
8.2 |
4.19.0+1 |
x86 |
✔ |
BFB Version Upgrade/Downgrade
The following table provides a matrix for the supported upgrade/downgrade of BFBs across different versions.
Version |
Upgrade to |
Downgrade to |
1.0.0 |
1.1.0; 1.1.1 |
N/A |
1.1.0 |
1.1.1; 1.2.0 |
1.0.0 |
1.1.1 |
1.2.0; 1.3.0 |
1.1.0; 1.0.0 |
1.2.0 |
1.3.0; 1.4.0 |
1.1.1; 1.1.0 |
1.3.0 |
1.4.0; 1.5.0 |
1.2.0; 1.1.1 |
1.4.0 |
1.5.0; 2.0.2 |
1.3.0; 1.2.0 |
1.5.0 |
2.0.2; 2.2.0; 1.5.1; 1.5.2; 1.5.3 |
1.4.0; 1.3.0 |
1.5.1 |
1.5.2 |
1.5.0 |
1.5.2 |
1.5.3 |
1.5.1; 1.5.0 |
1.5.3 |
N/A |
1.5.2; 1.5.0 |
2.0.2 |
2.2.0; 2.5.0 |
1.5.0; 1.4.0 |
2.2.0 |
2.5.0; 2.6.0 |
N/A |
2.2.1 |
2.5.0; 2.6.0 |
N/A |
2.5.0 |
2.5.1; 2.6.0 |
2.2.1 for BlueField-3; 2.2.0 for BlueField-2 |
2.5.1 |
N/A |
2.5.0 |
2.6.0 |
2.7.0 |
2.5.0; 2.2.1 for BlueField-3; 2.2.0 for BlueField-2 |
2.7.0 |
N/A |
2.6.0; 2.5.0; 2.2.1 for BlueField-3; 2.2.0 for BlueField-2 |
Supported DOCA Version Upgrade Using Standard Linux Tools on BlueField
Version |
Upgrade to |
2.5.0 |
2.5.1; 2.6.0 |
2.5.1 |
N/A |
2.6.0 |
2.7.0 |
Customers who purchased NVIDIA products directly from NVIDIA are invited to contact us through the following methods:
E-mail: enterprisesupport@nvidia.com
Enterprise Support page: https://www.nvidia.com/en-us/support/enterprise
Customers who purchased NVIDIA M-1 Global Support Services, please see your contract for details regarding Technical Support.
Customers who purchased NVIDIA products through an NVIDIA-approved reseller should first seek assistance through their reseller.
For questions, comments, and feedback, please contact us at DOCA-Feedback@exchange.nvidia.com.
The following table lists the known issues and limitations for this release of DOCA SDK.
Reference |
Description |
3863011 |
Description: Restarting openibd results in kernel panic after modifying kernel IPsec configurations. |
Workaround: N/A |
|
Keyword: IPsec |
|
Reported in version: 2.7.0 |
|
3882794 |
Description: When working with doca_pcc_np context, the return value from the API doca_pcc_get_max_num_threads() is incorrect. The function has an output parameter that indicates the maximum number of threads allowed for a doca_pcc_np context. The correct value that the library expects is 16 instead of the returned 64. |
Workaround: N/A |
|
Keyword: PCC; threads |
|
Reported in version: 2.7.0 |
|
3886674 |
Description: Installing doca-all and other DOCA metapackages does not install the mlnx-nvme driver. |
Workaround: mlnx-nvme is only needed for NVMe-over-RDMA remote storage support. If you wish to install it, add the mlnx-nvme package to the install command.
|
|
Keyword: NVMe; DOCA profile |
|
Reported in version: 2.7.0 |
|
3885930 |
Description: When installing DOCA-Host on a system using NVMe storage (typically local NVMe disk), and the script doca-kernel-support is used to rebuild and install kernel modules, unloading the mlx5 drivers is only possible after also unmounting the NVMe storage, which would typically necessitate a reboot. |
Workaround: N/A |
|
Keyword: NVMe; doca-kernel-support; DOCA for host |
|
Reported in version: 2.7.0 |
|
3886315 |
Description: To reset or shut down the BlueField Arm, it is mandatory to specify the --sync 0 argument with reset level 1 and reset type 3 or 4. For example:
|
Workaround: N/A |
|
Keyword: Arm; shutdown |
|
Reported in version: 2.7.0 |
|
3837255 |
Description: When running Arm shutdown from the host OS it is expected to get the message -E- Failed to send Register MRSI. This message should be ignored. |
Workaround: Wait 2 more minutes before rebooting the host. Before proceeding with host OS reboot, it is recommended to query the operational state of the BlueField Arm cores from the BlueField BMC to verify that shutdown state has been reached. Run the following command:
Expected output is "06". |
|
Keyword: Host OS; reboot; error |
|
Reported in version: 2.7.0 |
|
3881941 |
Description: When working with RShim 2.0.28, PCIe host crash may rarely occur at the beginning of BFB push after the Arm reset. |
Workaround: Downgrade to RShim 2.0.27 or upgrade to RShim 2.0.29. |
|
Keyword: RShim; driver |
|
Reported in version: 2.7.0 |
|
3844705 |
Description: In OpenEuler 20.03, the Linux Kernel version 4.19.90 is affected by an issue that impacts the discard/trim functionality for the BlueField eMMC device which may cause degraded performance of the BlueField eMMC over time. |
Workaround: Upgrade to Linux Kernel version 5.10 or later. |
|
Keyword: eMMC discard; trim functionality |
|
Reported in version: 2.7.0 |
|
3877725 |
Description: During BFB installation in NIC mode on BlueField-3, too much information is added into RShim log which fills it, causing the Linux installation progress log to not appear in the RShim log.
|
Workaround: Monitor the BlueField-3 Arm's UART console to check whether BFB installation has completed or not for NIC mode.
|
|
Keyword: NIC mode; BFB install |
|
Reported in version: 2.7.0 |
|
3855702 |
Description: Trying to jump from a steering level in the hardware to a lower level using software steering is not supported on rdma-core lower than 48.x. |
Workaround: N/A |
|
Keyword: RDMA; SWS |
|
Reported in version: 2.7.0 |
|
3855485 |
Description: When enabling the PCI_SWITCH_EMULATION_ENABLE NVconfig, the mlx devices, and potentially the RShim devices disappear. Also, looking at the kernel logs using dmesg shows the following messages:
|
Workaround: N/A |
|
Keyword: NVconfig; RShim; dmsg |
|
Reported in version: 2.7.0 |
|
3831230 |
Description: In OpenEuler 20.03, the Linux Kernel version 4.19.90 is affected by an issue that impacts the discard/trim functionality for BlueField eMMC device which may cause degraded performance of BlueField eMMC over time. |
Workaround: Upgrade to Linux Kernel version 5.10 or later. |
|
Keyword: eMMC discard; trim functionality |
|
Reported in version: 2.7.0 |
|
3743879 |
Description: mlxfwreset could timeout on servers where the RShim driver is running and INTx is not supported. The following error message is printed: BF reset flow encountered a failure due to a reset state error of negotiation timeout. |
Workaround: Set PCIE_HAS_VFIO=0 and PCIE_HAS_UIO=0 in /etc/rshim.conf and restart the RShim driver. Then re-run the mlxfwreset command. If host Linux kernel lockdown is enabled, then manually unbind the RShim driver before mlxfwreset and bind it back after mlxfwreset:
|
|
Keyword: Timeout; mlxfwreset; INTx |
|
Reported in version: 2.7.0 |
|
3869639 |
Description: Users c annot use --job-output-buffer-size 0 when using remote output memory (--use-remote-output-buffers). |
Workaround: |
|
Keyword: DOCA Bench |
|
Reported in version: 2.7.0 |
|
3872654 |
Description: And i ssue occurs when submitting tasks with DOCA SHA with the following error.
|
Workaround: Reattempt the submit u sing a different --data-provider-job-count value. This workaround may also fail. |
|
Keyword: DOCA Bench |
|
Reported in version: 2.7.0 |
|
3859823 |
Description: Multi-threaded tests using DOCA Comch may hang or emit an infinite amount of log messages. Single-threaded tests are less likely to cause this issue. |
Workaround: N/A |
|
Keyword: DOCA Bench; DOCA Comch |
|
Reported in version: 2.7.0 |
|
3857095 |
Description: Send tasks on DOCA RDMA may fail. |
Workaround: N/A |
|
Keyword: DOCA Bench; DOCA RDMA; send |
|
Reported in version: 2.7.0 |
|
3857097 |
Description: DOCA RDMA tests cannot be launched from BlueField side. |
Workaround: N/A |
|
Keyword: DOCA Bench; DOCA RDMA |
|
Reported in version: 2.7.0 |
|
3849701 |
Description: DOCA Comch tests can not be launched from BlueField side . |
Workaround: N/A |
|
Keyword: DOCA Bench; DOCA Comch |
|
Reported in version: 2.7.0 |
|
3840230 |
Description: Order of cores specified in --core-list is not respected. Cores are picked in ascending order instead. |
Workaround: N/A |
|
Keyword: DOCA Bench |
|
Reported in version: 2.7.0 |
|
3665070 |
Description: Virtio-net controller fails to load if DPA_AUTHENTICATION is enabled. |
Workaround: N/A |
|
Keyword: Virtio-net; DPA |
|
Reported in version: 2.5.0 |
|
3678069 |
Description: If using BlueField with NVMe and mmcbld and configured to boot from mmcblk, users must create bf.cfg file with device=/dev/mmcblk0, then install the *.bfb as normal. |
Workaround: N/A |
|
Keyword: NVMe |
|
Reported in version: 2.5.0 |
|
3680538 |
Description: When using strongSwan or OVS-IPsec as explained in the NVIDIA BlueField DPU BSP, the IPSec Rx data path is not offloaded to hardware and occurs in software running on the Arm cores. As a result, bandwidth performance is substantially low. |
Workaround: N/A |
|
Keyword: IPsec |
|
Reported in version: 2.5.0 |
|
N/A |
Description: Execution unit partitions are still not implemented and would be added in a future release. |
Workaround: N/A |
|
Keyword: EU tool |
|
Reported in version: 2.5.0 |
|
3666160 |
Description: Installing BFB using bfb-install when mlxconfig PF_TOTAL_SF>1700, triggers server reboot immediately. |
Workaround: Change PF_TOTAL_SF to 0, perform a graceful shutdown, power cycle, then installing BFB. |
|
Keyword: SF; PF_TOTAL_SF; BFB installation |
|
Reported in version: 2.2.1 |
|
3594836 |
Description: When enabling Flex IO SDK tracer at high rates, a slow-down in processing may occur and/or some traces may be lost. |
Workaround: Keep tracing limited to ~1M traces per second to avoid a significant processing slow-down. Use tracer for debug purposes and consider disabling it by default. |
|
Keyword: Tracer FlexIO |
|
Reported in version: 2.2.1 |
|
3592080 |
Description: When using UEK8 on the host in DPU mode, creating a VF on the host consumes about 100MB memory on BlueField |
Workaround: N/A |
|
Keyword: UEK; VF |
|
Reported in version: 2.2.1 |
|
3566042 |
Description: Virtio hotplug is not supported in GPU-HOST mode on the NVIDIA Converged Accelerator. |
Workaround: N/A |
|
Keyword: Virtio; Converged Accelerator |
|
Reported in version: 2.2.0 |
|
3546474 |
Description: PXE boot over ConnectX interface might not work due to an invalid MAC address in the UEFI boot entry. |
Workaround: On BlueField, create /etc/bf.cfg file with the relevant PXE boot entries, then run the command bfcfg. |
|
Keyword: PXE; boot; MAC |
|
Reported in version: 2.2.0 |
|
3561723 |
Description: Running mlxfwreset sync 1 on NVIDIA Converged Accelerators may be reported as supported although it is not. Executing the reset will fail. |
Workaround: N/A |
|
Keywords: mlxfwreset |
|
Reported in version: 2.2.0 |
|
3546202 |
Description: After rebooting a BlueField-3 DPU running Rocky Linux 8.6 BFB, the kernel log shows the following error:
This message indicates that the Ethernet driver will function normally in all aspects, except that PHY polling is enabled. |
Workaround: N/A |
|
Keywords: Linux; PHY; kernel |
|
Reported in version: 2.2.0 |
|
3306489 |
Description: When performing longevity tests (e.g., mlxfwreset, DPU reboot, burning of new BFBs), a host running an Intel CPU may observer errors related to "CPU 0: Machine Check Exception". |
Workaround: Add intel_idle.max_cstate=1 entry to the kernel command line. |
|
Keywords: Longevity; mlxfwreset; DPU reboot |
|
Reported in version: 2.2.0 |
|
3529297 |
Description: Enhanced NIC mode is not supported on BlueField-2. |
Workaround: N/A |
|
Keywords: Operation; mode |
|
Reported in version: 2.2.0 |
|
3538486 |
Description: When removing LAG configuration from BlueField, a kernel warning for uverbs_destroy_ufile_hw is observed if virtio-net-controller is still running. |
Workaround: Stop virtio-net-controller service before cleaning up bond configuration. |
|
Keywords: Virtio-net; LAG |
|
Reported in version: 2.2.0 |
|
3534219 |
Description: On BlueField-3 devices, from DOCA 2.2.0 to 32.37.1306 (or lower), the host crashes when executing partial Arm reset (e.g., Arm reboot; BFB push; mlxfwreset). |
Workaround: Before downgrading the firmware:
|
|
Keyword: BlueField-3; downgrade |
|
Reported in version: 2.2.0 |
|
3462630 |
When trying to perform a PXE installation when UEFI Secure Boot is enabled, the following error messages may be observed:
|
Workaround: Download a Grub EFI binary from the Ubuntu website. For further information on Ubuntu UEFI Secure Boot PXE Boot, please visit Ubuntu's official website. |
|
Keyword: PXE; UEFI Secure Boot |
|
Reported in version: 2.0.2 |
|
3448841 |
Description: While running CentOS 8.2, switchdev Ethernet BlueField runs in "shared" RDMA net namespace mode instead of "exclusive". |
Workaround: Use ib_core module parameter netns_mode=0. For example:
|
|
Keyword: RDMA; isolation; Net NS |
|
Reported in version: 2.0.2 |
|
2706803 |
Description: When an NVMe controller, SoC management controller, and DMA controller are configured, the maximum number of VFs is limited to 124. |
Workaround: N/A |
|
Keyword: VF; limitation |
|
Reported in version: 2.0.2 |
|
3273435 |
Description: Changing the mode of operation between NIC and DPU modes results in different capabilities for the host driver which might cause unexpected behavior. |
Workaround: Reload the host driver or reboot the host. |
|
Keyword: Modes of operation; driver |
|
Reported in version: 2.0.2 |
|
3264749 |
Description: In Rocky and CentOS 8.2 inbox-kernel BFBs, RegEx requires the following extra huge page configuration for it to function properly:
If these commands have executed successfully you should see active (running) in the last line of the output. |
Workaround: N/A |
|
Keyword: RegEx; hugepages |
|
Reported in version: 1.5.1 |
|
3240153 |
Description: DOCA kernel support only works on a non-default kernel. |
Workaround: N/A |
|
Keyword: Kernel |
|
Reported in version: 1.5.0 |
|
3217627 |
Description: The doca_devinfo_rep_list_create API returns success on the host instead of Operation not supported. |
Workaround: N/A |
|
Keyword: DOCA core; InfiniBand |
|
Reported in version: 1.5.0 |