When creating 1 send queue (SQ) and 1 receive queue (RQ), each representor consumes ~3MB memory per single channel. Scaling this to the desired 1024 representors (SFs and/or VFs) would require ~3GB worth of memory for single channel. A major chunk of the 3MB is contributed by RQ allocation (receive buffers and SKBs). Therefore, to make efficient use of memory, shared RQ mode is implemented so PF/VF/SF representors share receive queues owned by the uplink representor.
The feature is enabled by default. To disable it:
Edit the field ALLOW_SHARED_RQ in /etc/mellanox/mlnx-bf.conf as follows:
ALLOW_SHARED_RQ="no"
Restart the driver. Run:
/etc/init.d/openibd restart
To connect from the host to BlueField in shared RQ mode, please refer to section Verifying Connection from Host to BlueField.
PF/VF representor to PF/VF communication on the host is not possible.
The following behavior is observed in shared RQ mode:
It is expected to see a 0 in the rx_bytes and rx_packets and valid vport_rx_packets and vport_rx_bytes after running traffic. Example output:
# ethtool -S pf0hpf NIC statistics: rx_packets: 0 rx_bytes: 0 tx_packets: 66946 tx_bytes: 8786869 vport_rx_packets: 546093 vport_rx_bytes: 321100036 vport_tx_packets: 549449 vport_tx_bytes: 321679548
Ethtool usage – in this mode, it is not possible to change/set the ring or coalesce parameters for the RX side using ethtool. Changing channels also only affects the TX side.