RoCE v2 UDP Port

Linux Kernel Upstream Release Notes v6.5

In RoCEv2, the RDMA payload is encapsulated as UDP payload with a specific UDP destination port number indicating that the payload is RDMA.

Prior to WinOF v5.02v5.22, the destination port number indicating RoCEv2 traffic was 1021. As of WinOF v5.35, the default destination port number used is 4791. This is to comply with The Internet Assigned Numbers Authority (IANA) guidance.

The UDP destination port is a configurable parameter of the driver. For its registry key, please refer to the RoCE Options table.

Since the default RoCEv2 port is changed in WinOF 5.10.50000, upgrade from an older version that uses the RoCEv2 with the default port will effectively change the port used for RoCEv2. Therefore, on a system that uses an older version with RoCEv2 and the default port, when upgrading to Rev 5.10.50000 or newer, it is advised that the entire group of computers be upgraded at the same time in order to maintain RoCEv2 connectivity.

To allow gradual upgrade without affecting the RoCEv2 connectivity, it is possible to override the default port before upgrade. This can be done by setting the roce_udp_dport parameter to the desired port in the registry so that this port is used by both older and newer versions.

© Copyright 2023, NVIDIA. Last updated on Oct 26, 2023.