BlueField-3 Jumbo MTU Not Working

NVIDIA BlueField Virtio-net v1.9.0

On This Page

Ping failed with packet size greater than 1500/4000 after configuring jumbo MTU.

Jumbo MTU is supported starting from the following kernel version:

Release

Upstream

VM kernel: 4.18.0-193.el8.x86_64

VM Linux version supports big MTU after 4.11.

Ubuntu

DOCA_2.5.0_BSP_4.5.0_Ubuntu_22.04

Virtnet

v1.7 or v1.6.26

The following steps configure jumbo MTU:

  1. Change the MTU of uplink representor (or bond) from the BlueField Arm OS:

    Copy
    Copied!
                

    # echo 9216 > /sys/bus/pci/devices/0000:03:00.0/net/p0/mtu

  2. Restart virtio-net-controller from the BlueField Arm OS:

    Copy
    Copied!
                

    # systemctl restart virtio-net-controlle

  3. Change the corresponding device MTU on BlueField Arm OS. For example, for the first VF on the first PF, run:

    Copy
    Copied!
                

    # virtnet modify -p 0 -v 0 device -t 9216

  4. Reload the virtio driver from the guest OS:

    Copy
    Copied!
                

    # modprobe -rv virtio-net && modprobe -v virtio-net

  5. Verify the VQs’ MTU configuration is correct on BlueField Arm OS:

    Copy
    Copied!
                

    # virtnet query -p 0 -v 0 --dbg_stats | grep jumbo_mtu   "jumbo_mtu": 1   "jumbo_mtu": 1

  6. Change the MTU of the virtio-net interface from the guest OS:

    Copy
    Copied!
                

    # echo 9216 > /sys/bus/pci/devices/0000:af:00.2/virtio0/net/enp175s0f2/mtu

© Copyright 2024, NVIDIA. Last updated on Jun 18, 2024.