eSwitch
This page offers troubleshooting information for eSwitch users and customers.
|
Command |
Explanation |
|
|
Lists all devlink devices. |
|
|
Displays devlink device eSwitch attributes. Device format: |
|
|
Sets the device switch mode (legacy or switchdev). Mode can be |
|
|
Dumps all device configuration parameters. |
|
|
Enables or disables TC HW offload. |
|
|
Displays all devlink port attributes. |
|
|
Dumps representor statistics. For uplink representors, exposes additional hardware counters. |
|
|
Dumps representor driver information. |
|
|
Captures non-offloaded (slow path) packets on the representor. |
|
|
Dumps HW lag |
|
|
Dumps the number of HW offloaded connections. |
|
|
Dumps logs from the kernel, including mlx5 driver logs. |
|
|
Dumps extended hardware configuration information. |
Unable to Configure Legacy Mode
To set devlink to "Legacy" mode in BlueField, run:
# devlink dev eswitch set pci/0000:03:00.0 mode legacy
# devlink dev eswitch set pci/0000:03:00.1 mode legacy
Please verify that:
No virtual functions are open. To verify if VFs are configured, run:
# /opt/mellanox/iproute2/sbin/rdma link | grep -i up link mlx5_0/2 state ACTIVE physical_state LINK_UP netdev pf0vf0 link mlx5_1/2 state ACTIVE physical_state LINK_UP netdev pf1vf0
If any VFs are configured, destroy them by running:
# echo 0 > /sys/class/infiniband/mlx5_0/device/mlx5_num_vfs # echo 0 > /sys/class/infiniband/mlx5_1/device/mlx5_num_vfs
If any SFs are configured, delete them by running:
/sbin/mlnx-sf -a delete --sfindex <SF Index>
NoteYou may retrieve the <SF Index> of the currently installed SFs by running:
# mlnx-sf -a show SF Index: pci/0000:03:00.0/229408 Parent PCI dev: 0000:03:00.0 Representor netdev: en3f0pf0sf0 Function HWADDR: 02:61:f6:21:32:8c Auxiliary device: mlx5_core.sf.2 netdev: enp3s0f0s0 RDMA dev: mlx5_2 SF Index: pci/0000:03:00.1/294944 Parent PCI dev: 0000:03:00.1 Representor netdev: en3f1pf1sf0 Function HWADDR: 02:30:13:6a:2d:2c Auxiliary device: mlx5_core.sf.3 netdev: enp3s0f1s0 RDMA dev: mlx5_3
Pay attention to the SF Index values. For example:
/sbin/mlnx-sf -a delete --sfindex pci/0000:03:00.0/229408 /sbin/mlnx-sf -a delete --sfindex pci/0000:03:00.1/294944
If the error "Error: mlx5_core: Can't change mode when flows are configured" is encountered while trying to configure legacy mode, ensure the following:
- Any configured SFs are deleted (see above for commands).
Shut down the links of all interfaces, delete any ip xfrm rules, delete any configured OVS flows, and stop openvswitch service. Run:
ip link set dev p0 down ip link set dev p1 down ip link set dev pf0hpf down ip link set dev pf1hpf down ip link set dev vxlan_sys_4789 down ip x s f ; ip x p f ; tc filter del dev p0 ingress tc filter del dev p1 ingress tc qdisc show dev p0 tc qdisc show dev p1 tc qdisc del dev p0 ingress tc qdisc del dev p1 ingress tc qdisc show dev p0 tc qdisc show dev p1 systemctl stop openvswitch-switch
Arm Appears as Two Interfaces
Expected error:
# sudo /opt/mellanox/iproute2/sbin/rdma link
link mlx5_0/1 state ACTIVE physical_state LINK_UP netdev p0
link mlx5_1/1 state ACTIVE physical_state LINK_UP netdev p1
Check if you are working in legacy mode.
# devlink dev eswitch show pci/0000:03:00.<0|1>
If the following line is printed, this means that you are working in legacy mode:
pci/0000:03:00.<0|1>: mode legacy inline-mode none encap enable
Please configure the DPU to work in switchdev mode. Run:
devlink dev eswitch set pci/0000:03:00.<0|1> mode switchdev
Check if you are working in separated mode:
# mlxconfig -d /dev/mst/mt41686_pciconf0 q | grep -i cpu * INTERNAL_CPU_MODEL SEPERATED_HOST(0)
Please configure the DPU to work in embedded mode. Run:
mlxconfig -d /dev/mst/mt41686_pciconf0 s INTERNAL_CPU_MODEL=1