Priority Flow Control (PFC)

Priority Flow Control (PFC) provides an enhancement to the existing pause mechanism in Ethernet. The current Ethernet pause option stops all traffic on a link. PFC creates eight separate virtual links on the physical link and allows any of these links to be paused and restarted independently, enabling the network to create a no-drop class of service for an individual virtual link.

PFC offers the following features:

Provides per-priority enabling or disabling of flow control
Transmits PFC-PAUSE frames when the receive threshold for a particular traffic class is reached
Provides the management capability for an administrator to configure the flow control properties on each port of the switch
Keeps flow control disabled for all priorities on all ports by default
Allows an administrator to enable or disable flow control per port and per priority level
Supports flow control only on physical ports, not on logical interfaces such as tunnels or interfaces defined by sharing a physical port in multiple virtual switch contexts
Uses the configured threshold values to set up the queue buffer spaces accordingly in the data-path
Provides hardware abstraction layer call-outs for the following:
Enabling or disabling of flow control on each port for each priority
Configuring the queue depth for each priority on each port
Provides trace logs for execution upon error conditions and for any event notifications from the hardware or data-path. These trace logs are a useful aid in troubleshooting.
Allows the administrator to configure the minimum and maximum threshold values for flow control. These configurations are applied globally on all ports and priorities.

Priority Based Flow Control (PFC) provides an enhancement to the existing pause flow control mechanism as described in 802.1x.

To enable PFC globally,

Copy
Copied!

            
            switch (config) # dcb priority-flow-control enable
This action might cause traffic loss while shutting down a port with priority-flow-control mode on
Type 'yes' to confirm enable pfc globally: yes

To enable PFC per priority:

Enable PFC globally on the switch.

Copy
Copied!

            
            switch (config) # dcb priority-flow-control enable
This action might cause traffic loss while shutting down a port with priority-flow-control mode on
Type 'yes' to confirm enable pfc globally: yes

Choose the priority you want to enable.

Copy
Copied!

            
            switch (config) # dcb priority-flow-control priority 5 enable

To enable PFC per interface, do the following.

Enable PFC globally on the switch.

Copy
Copied!

            
            switch (config) # dcb priority-flow-control enable

Choose the priority you want to enable.

Copy
Copied!

            
            switch (config) # dcb priority-flow-control 5 enable

Change to Interface mode.

Copy
Copied!

            
            switch (config) # interface ethernet 1/1
switch (config interface ethernet 1/1) #

Enable PFC for the specific interface.

Copy
Copied!

            
            switch (config interface ethernet 1/1) # dcb priority-flow-control mode on

When working with lossless traffic, the receiving side sends a pause frame (Xoff) to the transmitting side before the buffer is filled. When the buffer empties, the receiving side sends an un-pause frame (Xon) to the transmitting side.

Flow Control Threshold Configuration

The user has to set the buffer usage Xoff and Xon thresholds. The thresholds depend on network parameters (bandwidth, link latency, MTU) and the allocated size for the region.

Warning

When working with global flow control mode only, a single PG shall be used and Xoff and Xon shall be set on this PG. When working with priority flow control, Xoff and Xon shall be set on each lossless PG.

Warning

See the “Shared Buffers” page for more information on flow control.

PFC Watchdog

Lossless networks with PFC enabled provide strong packet delivery guarantees. However, lossless networks introduce a new fault scenario where a queue of an end-port (e.g. the port of a host connected to the network) may not be able to receive any traffic from the network and keeps sending pause frames towards the switch. Since lossless switch paths do not drop packets but decline receiving more packets when their buffers fill up, if the end-port queue is stuck for a long time, the buffers fill up not only for the target switch, but also on all switches with problematic port queues in the traffic forwarding path. This leads to endless PFC pause frames, also called a PFC storm, being observed on all switch ports along the path to the traffic source.

PFC watchdog prevents congestion from spreading in such a case. When switches detect this situation on any TC queue, all the packets in the queue are flushed and new packets destined to the same queue are dropped as well until PFC storming is relieved.

For lossless networks with global flow control configured, we will face the same issue of global pause storm. To resolve this, global-flow-control-watchdog mode is supported.

Additional Reading and Use Cases

For more information about this feature and its potential applications, please refer to the following community post:

How to Enable PFC on Switches (Spectrum)

PFC Commands

dcb priority-flow-control enable

	dcb priority-flow-control enable [force] disable dcb priority-flow-control [force] no dcb priority-flow-control enable [force] Enables PFC globally on the switch. It is also possible to assign specific interface behavior in dcb priority-flow-control mode. The disable form of the command globally disables PFC on the switch when RoCE mode is set to lossless/semi-lossless. The no form of the command sets global PFC to the default value. See “Default” section below.
Syntax Description	force	Forces operation
Default	PFC is generally disabled. See “RoCE Parameters” for specific RoCE modes in which the default is enabled
Configuration Mode	config
History	3.1.0000
	3.3.0000	Updated example
	3.8.2100	Updated example Added "disable dcb priority-flow-control" command Changes the function of the no form of the command
	3.9.0500	Updated the description of the "disable" form of the command and added a note
Example	switch (config)# no roce switch (config)# no dcb priority-flow-control enable force switch (config)# show dcb priority-flow-control PFC: disabled switch (config)# dcb priority-flow-control enable This action might cause traffic loss while shutting down a port with priority-flow-control mode on Type 'yes' to confirm enable pfc globally: yes switch (config)# show dcb priority-flow-control PFC: enabled switch (config)# roce semi-lossless switch (config)# show dcb priority-flow-control PFC: enabled switch (config)# disable dcb priority-flow-control force switch (config)# show dcb priority-flow-control PFC: disabled switch (config)# no dcb priority-flow-control enable force switch (config)# show dcb priority-flow-control PFC: enabled
Related Commands	show dcb priority-flow-control dcb priority-flow-control mode
Notes	This command asks the user to approve traffic loss because some interfaces with DCB mode activated might get shut down. The disable command is valid only for roce lossless/semi-lossless modes. For explicitly disabling PFC on other scenarios, please set the interface PFC mode to 'off' for all required ports.

dcb priority-flow-control priority

	dcb priority-flow-control priority <prio> enable no dcb priority-flow-control priority <prio> enable Enables PFC per priority on the switch. The no form of the command disables PFC per priority on the switch.
Syntax Description	prio	0-7
Default	PFC is disabled for all priorities.
Configuration Mode	config
History	3.1.0000
History	3.9.0500	Added note
Example	switch (config)# dcb priority-flow-control priority 0 enable
Related Commands	show dcb priority-flow-control
Notes	When RoCE mode is set to lossless/semi-lossless, the no form of the command is not applicable. For explicitly disabling PFC, set the interface PFC mode to 'off' for all required ports.

dcb priority-flow-control mode

	dcb priority-flow-control mode <mode> [force] no dcb priority-flow-control mode [force] Changes PFC mode per interface. The no form of the command disables PFC per interface.
Syntax Description	force	Configures the PFC admin mode as on or auto with no confirmation needed if the port is admin enabled
Syntax Description	mode	The interface PFC mode. Possible values: on –enables PFC per interface off –disables PFC per interface auto –set PFC mode for the interface to be controlled with traffic pool configuration
Default	auto –PFC mode is established by traffic pool configuration (not a directly configurable mode)
Configuration Mode	config interface ethernet config interface port-channel config interface mlag-port-channel
History	3.1.0000
	3.3.4500	Added MPO configuration mode
	3.6.6000	Added “force” parameter
	3.6.6102	Added “mode” parameter
	3.6.7100	Updated “mode” parameter description
Example	switch (config interface ethernet 1/1) # dcb priority-flow-control mode on
Related Commands	show dcb priority-flow-control
Notes	For the “force” parameter, the no form of the command disables priority-flow-control without the preceding confirmation prompt For mode value “auto”, if a lossless traffic pool is configured, PFC is enabled for this port. Otherwise, PFC is disabled.

pfc-wd

	pfc-wd no pfc-wd Enables PFC watchdog on interface. The no form of the command disables PFC watchdog on interface.
Syntax Description	N/A
Default	Disabled
Configuration Mode	config interface ethernet config interface port-channel config interface mlag-port-channel
History	3.6.6000
Example	switch (config interface ethernet 1/1) # pfc-wd
Related Commands	show interface pfc-wd
Notes	When a user enables both "flowcontrol receive on" and "pfc-wd" on specific port, global-flow-control-watchdog mode is activated. If only "pfc-wd" is enabled, then the PFC-watchdog mode is activated.

show dcb priority-flow-control

	show dcb priority-flow-control [interface <type> <inf>] [detail] Displays DCB priority flow control configuration and status.
Syntax Description	type	ethernet port-channel
	inf	The interface number
	detail	Adds details information to the show output
Default	N/A
Configuration Mode	Any command mode
History	3.1.0000
Example	switch (config) # show dcb priority-flow-control PFC enabled Priority Enabled List : 0 Priority Disabled List : 1 2 3 4 5 6 7 TC Lossless --- ---------- 0 N 1 Y 2 Y 3 N Interface PFC admin PFC oper ------------ -------------- ------------- 1/1 On Enabled 1/2 Disabled Disabled 1/3 Disabled Disabled 1/4 Disabled Disabled ...
Related Commands
Notes

show dcb priority-flow-control interface mlag-port-channel

	show dcb priority-flow-control interface mlag-port-channel <inf> [detail] Displays DCB priority flow control configuration and status for MPO interfaces.
Syntax Description	inf	The interface number.
Syntax Description	detail	Adds details information to the show output.
Default	N/A
Configuration Mode	Any command mode
History	3.1.0000
History	3.6.6000	Updated example
Example	switch (config) # show dcb priority-flow-control interface mlag-port-channel 1 detail PFC: disabled Priority Enabled List: Priority Disabled List: 0 1 2 3 4 5 6 7 PFC Port Mpo1 Information: Port Mode : On Operational state : Off No Remote Entry is Present
Related Commands
Notes

show interface pfc-wd

	show interface <type> <id> pfc-wd Displays PFC watchdog information.
Syntax Description	type	Interface type: ethernet port-channel mlag-port-channel
Syntax Description	id	Interface ID
Default	N/A
Configuration Mode	Any command mode
History	3.6.6000
History	3.8.1300	Updated example
Example	switch (config) # show interfaces ethernet 1/1 pfc-wd Interface ethernet 1/1: PFC-WD admin : enable PFC-WD mode : global / per-priority / n/a Traffic Class 0 state: OK Traffic Class 1 state: OK Traffic Class 2 state: OK Traffic Class 3 state: OK Traffic Class 4 state: OK Traffic Class 5 state: OK Traffic Class 6 state: OK Traffic Class 7 state: OK switch (config) #
Related Commands	pfc-wd
Notes	When PFC-watchdog mode is activated, display "per-priority" in "PFC-WD mode". While global flow control watchdog activated, display "global". Otherwise, display "n/a".

On This Page