Known Issues

The following is a list of general limitations and known issues of the various components of the Inbox Driver release.

Internal Ref.

Issue

3344305

Description: Enabling both UPT VFs and SR-IOV VFs on the same host will result in IOMMU fault.
Workaround: N/A
Keywords: UPT SR-IOV
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

-

Description: Enabling VMDirectPath I/O pass-through of the first interface is not supported.
Workaround: N/A
Keywords: Pass-through
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

3308850

Description:

To create 126 UPT devices, the following configuration needs to be set manually:
# /opt/mellanox/bin/mlxconfig -d <dev> s NUM_OF_VFS=64
# /opt/mellanox/bin/mlxconfig -d <dev> s NUM_PF_MSIX=127

Workaround: N/A
Keywords: UPT
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

2678029

Description: Due to hardware limitations, Model 1 Level 2 and Model 2 for Enhanced Network Stack (ENS) mode in vSphere 8.0 is not supported in ConnectX-5 and ConnectX-6 adapter cards.
Workaround: Use ConnectX-6 Lx, ConnectX-6 Dx, or onwards cards that support ENS Model 1 Level 2 and Model 2A.
Keywords: ENS, ConnectX-5/ConnectX-6, Model 1 Level 2 and Model 2A
Adapter Cards / DPU: ConnectX-4 Onwards HCAs
Available in OS: ESXi 7.0u3, ESXi 7.0u2
Discovered in Version: 4.23.0.36

3192250

Description: The following error in logs may appear when performing DFW HW offload: "HW state is LAST_ACK while both sides sent an ACK".

This message is harmless and can be ignored.

Workaround: N/A
Keywords: DFW, offload, DPU
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

3038613

Description: On DPU, the following error message can appear upon driver loading: "ECPF is not EswManager, cannot get SF support on host".

This message is harmless and can be ignored.

Workaround: N/A
Keywords: Error, log
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

3175465

Description: The driver version in iDRAC is not displayed correctly.
Workaround: N/A
Keywords: iDRAC
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

-

Description: Geneve options length support is limited to 56B. Received packets with options length bigger than 56B are dropped.
Workaround: N/A
Keywords: Geneve
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

-

Description: The hardware can offload only up to 256B of headers.
Workaround: N/A
Keywords: Hardware offload
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

2204581

Description: A mismatch between the uplink and the VF MTU values may result in CQE with error.
Workaround: Align the uplink and the VF MTU values.
Keywords: CQE, error, model 2,
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

2429623

Description: Enabling sriov_mc_isolation module parameter may result in vmknic and emulated NICs multicast and IPv6 traffic loss.
Workaround: Unset or set the module parameter to 0.
Keywords: Multicast, IPv6, SR-IOV
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

2372060

Description: RDMA is not supported in the Hypervisor with ENS model 2.
Workaround: N/A
Keywords: ENS model 2, RDMA
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

2139469

Description: Setting the "Allow Guest MTU Change" option in vSphere Client is currently not functional. Although guest MTU changes in SR-IOV are allowed, they do not affect the port's MTU and the guest's MTU remains the same as the PF MTU.
Workaround: N/A
Keywords: MTU, SR-IOV
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

1340255

Description: ECN statistic counters accumulatorsPeriod and ecnMarkedRocePackets display wrong values and cannot be cleared.
Workaround: N/A
Keywords: nmlx5 ecn nmlxcli
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

1340275

Description: ECN tunable parameter initialAlphaValue for the Reaction Point protocol cannot be modified.
Workaround: N/A
Keywords: nmlx5 ecn nmlxcli
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

2430662

Description: Card's speed remains zero after port goes down and reboot is performed.
Workaround: Turn the port down and then up again.
Keywords: ConnectX-6 Dx, link speed
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

1514289

Description: RoCE traffic may fail after vMotion when using namespace.
Workaround: N/A
Keywords: Namespace, RoCE, vMotion
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

2334405

Description: Legacy SR-IOV is not supported with Model 1.
Workaround: Unset max_vfs or alternatively move to ENS model 0 or Model 2.
Keywords: SR-IOV, ENS
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

2449578

Description: When in ENS mode, changing the scheduler to HCLK, may cause traffic loss.
Workaround: N/A
Keywords: ENS, HCLK scheduler
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

746100

Description: The 'esxcli mellanox uplink link info -u <vmnic_name>' command reports the 'Auto negotiation' capability always as 'true'.
Workaround: N/A
Keywords: 'Auto negotiation' capability
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

1068621

Description: SMP MADs (ibnetdiscover, sminfo, iblinkinfo, smpdump, ibqueryerr, ibdiagnet and smpquery) are not supported on the VFs.
Workaround: N/A
Keywords: SMP MADs
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey, ConnectX-4 Onwards HCAs
Available in OS: ESXi 8.0, ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

1446060

Description: Although the max_vfs module parameter range is "0-128", due to firmware limitations, the following are the supported VFs per single port devices:
  • ConnectX-4 / ConnectX-5: up to 127

Workaround: N/A
Keywords: SR-IOV, VFs per port
Adapter Cards / DPU: ConnectX-4 Onwards HCAs
Available in OS: ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7
Discovered in Version: 4.23.0.36

2139469

Description: Setting the "Allow Guest MTU Change" option in vSphere Client is currently not functional. Although guest MTU changes in SR-IOV are allowed, they do not affect the port's MTU and the guest's MTU remains the same as the PF MTU.
Workaround: N/A
Keywords: MTU, SR-IOV
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

2813820

Description: The maximum supported VFs in the case of Model 3 (A and B) setup is 126.
Workaround: Set the maximum value of NUM_OF_VFS to 126 in mlxconfig.
Keywords: VFs, Model 3
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

2615605

Description: LAG creation fails when VFs are enabled on the host.
Workaround: Before adding a second uplink to the switch, make sure all VMs with VFs are powered off.
Keywords: Link Aggregation
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

2611342

Description: On older version of iDRAC, the NC-SI channel is not opened automatically.
Workaround: Run this command from the iDrac root SHEll manually and open the channel after every reboot or Arm OS

libncsitest 47 eth2 0 2 <MAC addr of sd0> <MAC addr of sd0>

Keywords: iDRAC Communication
Adapter Cards / DPU: NVIDIA BlueField-2 for VMware Monterey
Available in OS: ESXi 8.0
Discovered in Version: 4.23.0.36

-

Description: Adapter cards that come with a pre-configured link type as InfiniBand cannot be detected by the driver and cannot be seen by MFT tools. Thus its link type cannot be changed.
Workaround:
  1. Unload the driver.
    unload nmlx5_core module

  2. Make the device visible to MFT by loading the driver in a recovery mode.
    vmkload_mod nmlx5_core mst_recovery=1 kill the devmgr

  3. Check the device available on your machine.
    /opt/mellanox/bin/mst status

  4. Change the link type to Ethernet using MFT.
    opt/mellanox/bin/mlxconfig -d mt4115_pciconf0 set LINK_TYPE_P1=2 LINK_TYPE_P2=2

  5. Power Cycle the host.

Keywords: Link type, InfiniBand, MFT
Adapter Cards / DPU: ConnectX-4 Onwards HCAs
Available in OS: ESXi 7.0u3, ESXi 7.0u2, ESXi 7.0u1, ESXi 7.0, ESXi 6.7u2, ESXi 6.7

Discovered in versions: 4.17.13.10-vmw, 4.17.13.1-vmw, 4.17.16.8-vmw, 4.17.16.7-vmw, 4.17.9.12-vmw

Last updated on Sep 8, 2023.