Mergeable Rx Buffer
When negotiating with the driver, mergeable buffers is a mode where multiple descriptors are posted to fit a single jumbo sized packet coming from the wire. This is a receive-side only feature which helps im prove performance in situations of a large MTU (e.g., 9K).
Enabling and using mergeable buffers requires updating the configuration file along with advertising feature bits from the controller side as described in the following subsections.
To enable or disable the mergeable Rx buffer feature, set the mrg_rxbuf attribute in the virtnet.conf configuration file to 1 or 0 respectively.
For example, to enable mergeable Rx buffer:
[dpu]# cat /opt/mellanox/mlnx_virtnet/virtnet.conf
{
...
"mrg_rxbuf": 1
...
}
Updating the configuration file requires a restart of the virtio-net-controller.
Refer to "Configuration File" page for more information.
Mergeable buffer is a per-device feature.
Users must query a device to check if VIRTIO_F_MRG_RX_BUFFER is available. For example, the following PF 0 does not support mergeable buffer:
[dpu]# virtnet query -p 0 -b {'all': '0x0', 'pf': '0x0', 'dbg_stats': '0x0', 'brief': '0x1', 'latency_stats': '0x0', 'stats_clear': '0x0'} { "devices": [ { "pf_id": 0, "transitional": 0, "vuid": "MT2251X00020VNETS1D0F0", "pci_bdf": "86:00.0", "pci_dev_id": "0x1041", "pci_vendor_id": "0x1af4", "pci_class_code": "0x20000", "pci_subsys_id": "0x1", "pci_subsys_vendor_id": "0x1af4", "pci_revision_id": "1", "pci_max_vfs": "0", "enabled_vfs": "0", "device_feature": { "value": "0x8900010300e7182f", " 0": "VIRTIO_NET_F_CSUM", " 1": "VIRTIO_NET_F_GUEST_CSUM", " 2": "VIRTIO_NET_F_CTRL_GUEST_OFFLOADS", " 3": "VIRTIO_NET_F_MTU", " 5": "VIRTIO_NET_F_MAC", " 11": "VIRTIO_NET_F_HOST_TSO4", " 12": "VIRTIO_NET_F_HOST_TSO6", " 16": "VIRTIO_NET_F_STATUS", " 17": "VIRTIO_NET_F_CTRL_VQ", " 18": "VIRTIO_NET_F_CTRL_RX", " 21": "VIRTIO_NET_F_GUEST_ANNOUNCE", " 22": "VIRTIO_NET_F_MQ", " 23": "VIRTIO_NET_F_CTRL_MAC_ADDR", " 32": "VIRTIO_F_VERSION_1", " 33": "VIRTIO_F_IOMMU_PLATFORM", " 40": "VIRTIO_F_RING_RESET", " 56": "VIRTIO_NET_F_HOST_USO", " 59": "VIRTIO_NET_F_GUEST_HDRLEN", " 63": "VIRTIO_NET_F_SPEED_DUPLEX" }, ... }
To enable the feature:
Make sure there is no driver loaded from the guest-OS side:
[host]# modprobe -rv virtio_net && modprobe -rv virtio_pci
Set the 15th bit to 1 in the feature bits, and modify the device:
[dpu]# virtnet modify -p 0 device -f 0x8900010300e7982f {'pf': '0x0', 'all': '0x0', 'subcmd': '0x0', 'features': '0x8900010300e7982f'} { "errno": 0, "errstr": "Success" }
Load the drivers from the host:
[host]# modprobe -v virtio_pci && modprobe -v virtio_net
Query the device again, checking whether VIRTIO_F_MRG_RX_BUFFER is available. The following query shows VIRTIO_F_MRG_RX_BUFFER under device_feature and driver_feature. Now mergeable buffer is enabled on PF 0.
[dpu]# virtnet query -p 0 -b {'all': '0x0', 'pf': '0x0', 'dbg_stats': '0x0', 'brief': '0x1', 'latency_stats': '0x0', 'stats_clear': '0x0'} { "devices": [ { "pf_id": 0, "transitional": 0, "vuid": "MT2251X00020VNETS0D0F1", "pci_bdf": "85:00.1", "pci_dev_id": "0x1041", "pci_vendor_id": "0x1af4", "pci_class_code": "0x20000", "pci_subsys_id": "0x1041", "pci_subsys_vendor_id": "0x1af4", "pci_revision_id": "1", "pci_max_vfs": "0", "enabled_vfs": "0", "device_feature": { "value": "0x8900032300e7982f", " 0": "VIRTIO_NET_F_CSUM", " 1": "VIRTIO_NET_F_GUEST_CSUM", " 2": "VIRTIO_NET_F_CTRL_GUEST_OFFLOADS", " 3": "VIRTIO_NET_F_MTU", " 5": "VIRTIO_NET_F_MAC", " 11": "VIRTIO_NET_F_HOST_TSO4", " 12": "VIRTIO_NET_F_HOST_TSO6", " 15": "VIRTIO_F_MRG_RX_BUFFER", " 16": "VIRTIO_NET_F_STATUS", " 17": "VIRTIO_NET_F_CTRL_VQ", " 18": "VIRTIO_NET_F_CTRL_RX", " 21": "VIRTIO_NET_F_GUEST_ANNOUNCE", " 22": "VIRTIO_NET_F_MQ", " 23": "VIRTIO_NET_F_CTRL_MAC_ADDR", " 32": "VIRTIO_F_VERSION_1", " 33": "VIRTIO_F_IOMMU_PLATFORM", " 37": "VIRTIO_F_SR_IOV", " 40": "VIRTIO_F_RING_RESET", " 41": "VIRTIO_F_ADMIN_VQ", " 56": "VIRTIO_NET_F_HOST_USO", " 59": "VIRTIO_NET_F_GUEST_HDRLEN", " 63": "VIRTIO_NET_F_SPEED_DUPLEX" }, "driver_feature": { "value": "0x8000002300e7982f", " 0": "VIRTIO_NET_F_CSUM", " 1": "VIRTIO_NET_F_GUEST_CSUM", " 2": "VIRTIO_NET_F_CTRL_GUEST_OFFLOADS", " 3": "VIRTIO_NET_F_MTU", " 5": "VIRTIO_NET_F_MAC", " 11": "VIRTIO_NET_F_HOST_TSO4", " 12": "VIRTIO_NET_F_HOST_TSO6", " 15": "VIRTIO_F_MRG_RX_BUFFER", " 16": "VIRTIO_NET_F_STATUS", " 17": "VIRTIO_NET_F_CTRL_VQ", " 18": "VIRTIO_NET_F_CTRL_RX", " 21": "VIRTIO_NET_F_GUEST_ANNOUNCE", " 22": "VIRTIO_NET_F_MQ", " 23": "VIRTIO_NET_F_CTRL_MAC_ADDR", " 32": "VIRTIO_F_VERSION_1", " 33": "VIRTIO_F_IOMMU_PLATFORM", " 37": "VIRTIO_F_SR_IOV", " 63": "VIRTIO_NET_F_SPEED_DUPLEX" }, ... }
The number of descriptors per work queue entry depends on the MTU size. For best performance, it is recommended to not enable the feature if the MTU is set to the default value (1500).
Performance is expected to degrade with this feature when receiving small sized packets (e.g., 64 bytes) from the wire.
Mergeable buffer does not work with the packed VQ feature.