VXLAN Hardware Stateless Offloads
VXLAN (Virtual Extensible LAN) technology addresses scalability and isolation challenges in modern data center networks by encapsulating Layer 2 frames within Layer 3 packets. However, this encapsulation introduces performance challenges that require extending traditional stateless offload capabilities to VXLAN traffic.
The ConnectX® family of adapter cards provides hardware acceleration for VXLAN-encapsulated packets, offering the same stateless offloads typically available for non-encapsulated traffic. VXLAN packets are encapsulated using an outer UDP header.
The following stateless offloads are supported for VXLAN traffic:
Checksum Generation
Inner IP checksum
Inner TCP/UDP checksum
Checksum Validation
Inner IP checksum
Inner TCP/UDP checksum
TSO (TCP Segmentation Offload)
Supported for inner TCP payloads
RSS (Receive Side Scaling)
Flow distribution based on inner packet attributes (e.g., IP addresses, TCP/UDP ports)
Receive Queue Selection
Inner frames can be steered to specific Queue Pairs (QPs) for advanced traffic steering and isolation
VXLAN offload is enabled by default for ConnectX-4 family devices running the minimum required firmware version and a kernel version that includes VXLAN support.
To confirm if the current setup supports VXLAN, run:
ethtool -k $DEV | grep udp_tnl
Example:
ethtool -k ens1f0 | grep udp_tnl
tx-udp_tnl-segmentation: on
ConnectX-4 family devices support configuring multiple UDP ports for VXLAN offload. Ports can be added to the device by configuring a VXLAN device from the OS command line using the ip command.
If you configure multiple UDP ports for offload and exceed the total number of ports supported by hardware, then those additional ports will still function properly, but will not benefit from any of the stateless offloads.
Example:
ip link add vxlan0 type vxlan id 10 group 239.0.0.10 ttl 10 dev ens1f0 dstport 4789
ip addr add 192.168.4.7/24 dev vxlan0
ip link set up vxlan0
dstport' parameters are not supported in Ubuntu 14.4.
The VXLAN ports can be removed by deleting the VXLAN interfaces.
Example:
ip link delete vxlan0
VXLAN tunneling adds an encapsulation overhead of 50 bytes per packet, comprising:
14 bytes – Ethernet header
20 bytes – Outer IP header
8 bytes – UDP header
8 bytes – VXLAN header
This overhead must be accounted for to prevent packet fragmentation or drops.
VXLAN adds 50 bytes of encapsulation overhead. To avoid fragmentation or packet loss, you must account for this overhead by adjusting the MTU:
Option 1: Decrease the MTU on the sending interface (e.g., VM virtio-net, host veth). For example: Use MTU 1450 instead of 1500.
Option 2: Increase the MTU on the physical uplink. For example: Use MTU 1550 instead of 1500.