Explicit Congestion Notification (ECN)

NVIDIA MLNX_EN Documentation Rev 5.4-3.7.5.0 LTS

ECN is an extension to the IP protocol. It allows reliable communication by notifying all ends of communication when congestion occurs. This is done without dropping packets.
Please note that this feature requires all nodes in the path (nodes, routers etc) between the communicating nodes to support ECN to ensure reliable communication. ECN is marked as 2 bits in the traffic control IP header. This ECN implementation refers to RoCE v2.

To enable ECN on the hosts:

  1. Enable ECN in sysfs.

    Copy
    Copied!
                

    /sys/class/net/<interface>/<protocol>/ecn_<protocol>_enable =1

  2. Query the attribute.

    Copy
    Copied!
                

    cat /sys/class/net/<interface>/ecn/<protocol>/params/<requested attribute>

  3. Modify the attribute.

    Copy
    Copied!
                

    echo <value> /sys/class/net/<interface>/ecn/<protocol>/params/<requested attribute>

ECN supports the following algorithms:

  • r_roce_ecn_rp - Reaction point

  • r_roce_ecn_np - Notification point

Each algorithm has a set of relevant parameters and statistics, which are defined per device, per port, per priority.

Copy
Copied!
            

cat /sys/class/net/<interface>/ecn/<protocol>/enable/X

Copy
Copied!
            

cat /sys/class/net/<interface>/ecn/<protocol>/requested attributes

Copy
Copied!
            

echo 1 > /sys/class/net/<interface>/ecn/<protocol>/enable/X

Copy
Copied!
            

echo <value> > /sys/class/net/<interface>/ecn/<protocol>/requested attributes

where:

  • X: priority {0..7}

  • protocol: roce_rp / roce_np

  • requested attributes: Next Slide for each protocol.

© Copyright 2023, NVIDIA. Last updated on Sep 8, 2023.