Shared RQ Mode

When creating 1 send queue (SQ) and 1 receive queue (RQ), each representor consumes ~3MB memory per single channel. Scaling this to the desired 1024 representors (SFs and/or VFs) would require ~3GB worth of memory for single channel. A major chunk of the 3MB is contributed by RQ allocation (receive buffers and SKBs). Therefore, to make efficient use of memory, shared RQ mode is implemented so PF/VF/SF representors share receive queues owned by the uplink representor.

The feature is enabled by default. To disable it:

1. Edit the field ALLOW_SHARED_RQ in /etc/mellanox/mlnx-bf.conf as follows:

   ALLOW_SHARED_RQ="no"

2. Restart the driver. Run:

   /etc/init.d/openibd restart

To connect from the host to BlueField in shared RQ mode, please refer to section Verifying Connection from Host to BlueField.

The following behavior is observed in shared RQ mode:

• It is expected to see a 0 in the rx_bytes and rx_packets and valid vport_rx_packets and vport_rx_bytes after running traffic. Example output:

  # ethtool -S pf0hpf
  NIC statistics:
       rx_packets: 0
       rx_bytes: 0
       tx_packets: 66946
       tx_bytes: 8786869
       vport_rx_packets: 546093
       vport_rx_bytes: 321100036
       vport_tx_packets: 549449
       vport_tx_bytes: 321679548

• Ethtool usage – in this mode, it is not possible to change/set the ring or coalesce parameters for the RX side using ethtool. Changing channels also only affects the TX side.