Priority Flow Control (PFC)
Priority Flow Control (PFC) provides an enhancement to the existing pause mechanism in Ethernet. The current Ethernet pause option stops all traffic on a link. PFC creates eight separate virtual links on the physical link and allows any of these links to be paused and restarted independently, enabling the network to create a no-drop class of service for an individual virtual link.
PFC offers the following features:
Provides per-priority enabling or disabling of flow control
Transmits PFC-PAUSE frames when the receive threshold for a particular traffic class is reached
Provides the management capability for an administrator to configure the flow control properties on each port of the switch
Keeps flow control disabled for all priorities on all ports by default
Allows an administrator to enable or disable flow control per port and per priority level
Supports flow control only on physical ports, not on logical interfaces such as tunnels or interfaces defined by sharing a physical port in multiple virtual switch contexts
Uses the configured threshold values to set up the queue buffer spaces accordingly in the data-path
Provides hardware abstraction layer call-outs for the following:
Enabling or disabling of flow control on each port for each priority
Configuring the queue depth for each priority on each port
Provides trace logs for execution upon error conditions and for any event notifications from the hardware or data-path. These trace logs are a useful aid in troubleshooting.
Allows the administrator to configure the minimum and maximum threshold values for flow control. These configurations are applied globally on all ports and priorities.
Priority Based Flow Control (PFC) provides an enhancement to the existing pause flow control mechanism as described in 802.1x.
To enable PFC globally,
switch
(config) # dcb priority-flow-control enable
This action might cause traffic loss while
shutting down a port with priority-flow-control mode on
Type 'yes'
to confirm enable pfc globally: yes
To enable PFC per priority:
Enable PFC globally on the switch.
switch
(config) # dcb priority-flow-control enable This action might cause traffic losswhile
shutting down a port with priority-flow-control mode on Type'yes'
to confirm enable pfc globally: yesChoose the priority you want to enable.
switch
(config) # dcb priority-flow-control priority5
enable
To enable PFC per interface, do the following.
Enable PFC globally on the switch.
switch
(config) # dcb priority-flow-control enableChoose the priority you want to enable.
switch
(config) # dcb priority-flow-control5
enableChange to Interface mode.
switch
(config) #interface
ethernet1
/1
switch
(configinterface
ethernet1
/1
) #Enable PFC for the specific interface.
switch
(configinterface
ethernet1
/1
) # dcb priority-flow-control mode on
When working with lossless traffic, the receiving side sends a pause frame (Xoff) to the transmitting side before the buffer is filled. When the buffer empties, the receiving side sends an un-pause frame (Xon) to the transmitting side.
The user has to set the buffer usage Xoff and Xon thresholds. The thresholds depend on network parameters (bandwidth, link latency, MTU) and the allocated size for the region.
When working with global flow control mode only, a single PG shall be used and Xoff and Xon shall be set on this PG. When working with priority flow control, Xoff and Xon shall be set on each lossless PG.
See the “Shared Buffers” page for more information on flow control.
Lossless networks with PFC enabled provide strong packet delivery guarantees. However, lossless networks introduce a new fault scenario where a queue of an end-port (e.g. the port of a host connected to the network) may not be able to receive any traffic from the network and keeps sending pause frames towards the switch. Since lossless switch paths do not drop packets but decline receiving more packets when their buffers fill up, if the end-port queue is stuck for a long time, the buffers fill up not only for the target switch, but also on all switches with problematic port queues in the traffic forwarding path. This leads to endless PFC pause frames, also called a PFC storm, being observed on all switch ports along the path to the traffic source.
PFC watchdog prevents congestion from spreading in such a case. When switches detect this situation on any TC queue, all the packets in the queue are flushed and new packets destined to the same queue are dropped as well until PFC storming is relieved.
For lossless networks with global flow control configured, we will face the same issue of global pause storm. To resolve this, global-flow-control-watchdog mode is supported.
For more information about this feature and its potential applications, please refer to the following community post:
dcb priority-flow-control enable
dcb priority-flow-control enable [force] disable dcb priority-flow-control [force] no dcb priority-flow-control enable [force] Enables PFC globally on the switch. It is also possible to assign specific interface behavior in dcb priority-flow-control mode. The disable form of the command globally disables PFC on the switch when RoCE mode is set to lossless/semi-lossless. The no form of the command sets global PFC to the default value. See “Default” section below. |
||
Syntax Description |
force |
Forces operation |
Default |
PFC is generally disabled. See “RoCE Parameters” for specific RoCE modes in which the default is enabled |
|
Configuration Mode |
config |
|
History |
3.1.0000 |
|
3.3.0000 |
Updated example |
|
3.8.2100 |
|
|
3.9.0500 |
Updated the description of the "disable" form of the command and added a note |
|
Example |
switch (config)# no roce |
|
Related Commands |
show dcb priority-flow-control dcb priority-flow-control mode |
|
Notes |
|
dcb priority-flow-control priority
dcb priority-flow-control priority <prio> enable no dcb priority-flow-control priority <prio> enable Enables PFC per priority on the switch. The no form of the command disables PFC per priority on the switch. |
||
Syntax Description |
prio |
0-7 |
Default |
PFC is disabled for all priorities. |
|
Configuration Mode |
config |
|
History |
3.1.0000 |
|
3.9.0500 |
Added note |
|
Example |
switch (config)# dcb priority-flow-control priority 0 enable |
|
Related Commands |
show dcb priority-flow-control |
|
Notes |
When RoCE mode is set to lossless/semi-lossless, the no form of the command is not applicable. For explicitly disabling PFC, set the interface PFC mode to 'off' for all required ports. |
dcb priority-flow-control mode
dcb priority-flow-control mode <mode> [force] no dcb priority-flow-control mode [force] Changes PFC mode per interface. The no form of the command disables PFC per interface. |
||
Syntax Description |
force |
Configures the PFC admin mode as on or auto with no confirmation needed if the port is admin enabled |
mode |
The interface PFC mode. Possible values:
|
|
Default |
auto – PFC mode is established by traffic pool configuration (not a directly configurable mode) |
|
Configuration Mode |
config interface ethernet config interface port-channel config interface mlag-port-channel |
|
History |
3.1.0000 |
|
3.3.4500 |
Added MPO configuration mode |
|
3.6.6000 |
Added “force” parameter |
|
3.6.6102 |
Added “mode” parameter |
|
3.6.7100 |
Updated “mode” parameter description |
|
Example |
switch (config interface ethernet 1/1) # dcb priority-flow-control mode on |
|
Related Commands |
show dcb priority-flow-control |
|
Notes |
|
pfc-wd
pfc-wd no pfc-wd Enables PFC watchdog on interface. The no form of the command disables PFC watchdog on interface. |
||
Syntax Description |
N/A |
|
Default |
Disabled |
|
Configuration Mode |
config interface ethernet config interface port-channel config interface mlag-port-channel |
|
History |
3.6.6000 |
|
Example |
switch (config interface ethernet 1/1) # pfc-wd |
|
Related Commands |
show interface pfc-wd |
|
Notes |
When a user enables both "flowcontrol receive on" and "pfc-wd" on specific port, global-flow-control-watchdog mode is activated. If only "pfc-wd" is enabled, then the PFC-watchdog mode is activated. |
show dcb priority-flow-control
show dcb priority-flow-control [interface <type> <inf>] [detail] Displays DCB priority flow control configuration and status. |
||
Syntax Description |
type |
|
inf |
The interface number |
|
detail |
Adds details information to the show output |
|
Default |
N/A |
|
Configuration Mode |
Any command mode |
|
History |
3.1.0000 |
|
Example |
switch (config) # show dcb priority-flow-control |
|
Related Commands |
||
Notes |
show dcb priority-flow-control interface mlag-port-channel
show dcb priority-flow-control interface mlag-port-channel <inf> [detail] Displays DCB priority flow control configuration and status for MPO interfaces. |
||
Syntax Description |
inf |
The interface number. |
detail |
Adds details information to the show output. |
|
Default |
N/A |
|
Configuration Mode |
Any command mode |
|
History |
3.1.0000 |
|
3.6.6000 |
Updated example |
|
Example |
switch (config) # show dcb priority-flow-control interface mlag-port-channel 1 detail |
|
Related Commands |
||
Notes |
show interface pfc-wd
|
show interface <type> <id> pfc-wd Displays PFC watchdog information. |
|
Syntax Description |
type |
Interface type:
|
id |
Interface ID |
|
Default |
N/A |
|
Configuration Mode |
Any command mode |
|
History |
3.6.6000 |
|
3.8.1300 |
Updated example |
|
Example |
switch (config) # show interfaces ethernet 1/1 pfc-wd Interface ethernet 1/1: PFC-WD admin : enable PFC-WD mode : global / per-priority / n/a Traffic Class 0 state: OK Traffic Class 1 state: OK Traffic Class 2 state: OK Traffic Class 3 state: OK Traffic Class 4 state: OK Traffic Class 5 state: OK Traffic Class 6 state: OK Traffic Class 7 state: OK switch (config) # |
|
Related Commands |
pfc-wd |
|
Notes |
When PFC-watchdog mode is activated, display "per-priority" in "PFC-WD mode". While global flow control watchdog activated, display "global". Otherwise, display "n/a". |