OpenFlow 1.3 Workflow
The OpenFlow (OF) pipeline is deployed in parallel to the usual NVIDIA Onyx pipeline.
The ingress port must be deployed in hybrid mode so as to serve both the OF and normal NVIDIA Onyx pipeline.
The ingress packet, which passes the VLAN and Spanning Tree filters and is a match to the user ACL table, either progresses to the regular NVIDIA Onyx flow or the OpenFlow pipeline depending on the port coupling.
The following table presents a general summary of the capabilities of the OpenFlow 1.3 pipeline. They are also described further on in the document.
Table |
Match |
Actions |
Group |
Meters |
ACLs [0-249] |
(Dynamic key) (Arbitrary mask) |
|
|
|
FDB [250] |
(Exact match) |
|
Select—{Weights} Output ports (without LAG) |
N/A |
Router [251] |
(LPM) |
(Must have DEC_TTL and SET_DMAC when output action is implemented) |
N/A |
An Access Control List (ACL) is a list of permissions attached to an object, to filter or match switches packets. When the pattern is matched at the hardware lookup engine, a specified action (e.g. permit/deny) is applied. The rule fields represent flow characteristics such as source and destination addresses, protocol and VLAN ID.
ACL support currently allows actions of permit or deny rules, and supports only ingress direction. ACL search pattern can be taken from either L2 or L3 fields.
Supported ACL Matching Rules
Ingress packets, arriving the ACL, are matched against any combination of the following parameters (defined as the key):
OXM_OF_METADATA—matches according to metadata
OXM_OF_IN_PORT—matches according to ingress port (exact match or wildcard)
OXM_OF_ETH_SRC—matches source MAC address
OXM_OF_ETH_DST—matches destination MAC address
OXM_OF_ETH_TYPE—matches EtherType
WarningWhen match rule is set to match eth_type 9100, VLAN ID matching does not work.
OXM_OF_VLAN_VID—matches VLAN ID
OXM_OF_VLAN_PCP—matches priority level
OXM_OF_IPV4_SRC—matches source IPv4 address
OXM_OF_IPV4_DST—matches destination IPv4 address
OXM_OF_IPV6_SRC—matches source IPv6 address
OXM_OF_IPV6_DST—matches destination IPv6 address
OXM_OF_IPV6_ND_TARGET
WarningOXM_OF_IPV6_ND_TARGET match rule is not supported.
OXM_OF_IP_PROTO—matches IP protocols (exact match or wildcard)
OXM_OF_IP_DSCP—matches IP DSCP field (exact match or wildcard)
OXM_OF_IP_ECN—matches network ECN (exact match or wildcard)
OXM_OF_NW_TTL—matches network TTL (exact match or wildcard)
OXM_OF_TCP_SRC—matches source TCP
OXM_OF_TCP_DST—matches destination TCP
OXM_OF_UDP_SRC—matches source UDP
OXM_OF_UDP_DST—matches destination UDP
OXM_OF_SCTP_SRC—matches source SCTP
OXM_OF_SCTP_DST—matches destination SCTP
OXM_OF_ICMPV4_TYPE—matches ICMP type
OXM_OF_ICMPV4_CODE—matches ICMP code
OXM_OF_ARP_OP—matches ARP OP code
OXM_OF_ARP_SPA—matches sender protocol address
OXM_OF_ARP_TPA—matches target protocol address
There is a default set of match keys configured. To see what it is, please run the command “show openflow table match-keys” on your machine. To alter it, please use the command “openflow table match-keys”.
Non-standard Matches
OpenFlow 1.3 is able to match non-standard OpenFlow matching rules by mapping them to standard ones. The following non-standard matches are supported:
Matching source/destination IPv4 address encapsulated with MPLS labels (up to 6 MPLS labels can be skipped)—ip_src_inner/ip_dst_inner is mapped to OXM_OF_IPV4_SRC, OXM_OF_IPV4_DST
Table configuration:
openflow table
0
match-keys dl_dst dl_src dl_type mpls_label vlan_vid openflow table10
match-keys ignr_eth_type ip_dst_inner ip_src_innerThe ignr_eth_type is needed to ignore the Ethertype of IP that is required by OpenFlow to set to as a prerequisite to match on IP addresses.
Rules:
openflow add-flows
1
table=0
,mpls,mpls_label:32
,actions=goto_table=10
openflow add-flows2
table=10
,ip,nw_src=10.10
.10.0
/24
,nw_dst=10.10
.20.0
/24
, actions=output:127
The above matches IP address from 10.10.10.0/24 to 10.10.20.0/24 which have MPLS label 32 as the first label.
Control actions are not supported for non-standard matches.
Supported Rule Table Instructions
The intercepted packet is processed according to the instructions on the rule tables. The supported instructions are as follows:
DROP—drops packet
OFPIT_GOTO_TABLE—sends the packet for processing by another rule table
OFPIT_METER—policer function; drops packet if it exceeds kbps/pktps limit
OFPIT_WRITE_METADATA—writes meta-data with mask <METADATA>/0xFFF
OFPIT_EXPERIMENTE—sends the packet for processing by another controller
OFPIT_APPLY_ACTIONS—applies certain actions specified in the section below
Supported ACL Apply Actions
The following actions are applied on ingress packets once a match is achieved on the ACL table:
OFPAT_OUTPUT—the packet is sent out to a port (may also be a controller port)
OFPAT_GROUP—the packet is sent out to a group
3 types of group ports are supported:All: The packet is broadcasted on all ports which are part of the defined group
Selected: The packets are distributed toward the group ports according to a weight mechanism
Fast-Failover (FF): FF is a group of ports, one of which is defined as the primary port through which the packets are transported. In a failure scenario (defined as part of the group definition), traffic becomes transported through the most eligible backup port (from the list of backup ports). Once the failure scenario ends, traffic is routed again through the primary port
OFPAT_POP_VLAN—strips 802.1Q (VLAN) tag from the packet
OFPAT_PUSH_VLAN—adds 802.1Q (VLAN) tag from the packet
OFPAT_SET_NW_TTL—modifies network TTL
OFPAT_DEC_NW_TTL—decrements network TTL
OFPAT_SET_FIELD—ACL set fields detailed in section below
Normal
Supported ACL Set Fields
The following modifications may be implemented on ingress packets:
OXM_OF_ETH_SRC—sets the source MAC address of the packet
OXM_OF_ETH_DST—sets the destination MAC address of the packet
OXM_OF_VLAN_VID—sets the VLAN ID of the packet
OXM_OF_VLAN_PCP—sets the VLAN priority code point (PCP; 0-7)
OXM_OF_IP_DSCP—sets IP DSCP
OXM_OF_IP_ECN—sets network ECN
NXM_NX_CT_NW_SRC*—sets the source IP address of the packet
NXM_NX_CT_NW_DST*—sets the destination IP address of the packet
NXM_NX_CT_TP_SRC*—sets the source L4 port of the packet
NXM_NX_CT_TP_DST*—sets the destination L4 port of the packet
*Supported only on Spectrum-2 and Spectrum-3 systems.
Supported ACL Meters
ACL tables support up to 968 meters with 1 band (drop) per meter.
Valid meter ID range: 1-969
Only the rate or the burst size fields can be modified using OFPMC_MODIFY
OFPMF_BURST meter type can be OFPMF_KBPS (KB/s) or OFPMF_PKTPS (number of packets per second) but not both
Meter actions:
OFPMBT_DROP—drops packet according to meter configuration
FDB Table (250)
The FDB table is the same one shared with regular NVIDIA Onyx configuration (e.g., learning, static macs, and so forth). The cumulative number of supported FDB rules is 88KB. FDB may only configure rules with priority of 0x8000. Hard timeout is supported for FDB table rules. FDB rules cannot have wildcard on VID/ETH_DST.
The default action for the FDB table is normal and this cannot be changed by the user.
Supported FDB Apply Actions
OFPAT_OUTPUT—the packet is sent out to a port (may be controller port)
DROP—drops packet
Normal
Supported FDB Matching Rules
OXM_OF_VLAN_VID—matches VLAN ID
OXM_OF_ETH_DST—matches destination MAC address
The OpenFlow router table and the regular NVIDIA Onyx router table share the same hardware resources, but are separated logically.
The cumulative number of supported FDB & router rules is 88K. Hard timeout, where the switch removes a rule after a configured timer expires, is supported for router table rules. Switch systems ignore rule priority and configure rules according to masklen in DST IPv4/IPv6 match. A rule with action output must have SET_FIELD with ETH_DST and DEC_NW_TTL. The default action for the router table is DROP.
Set DMAC can be assigned only to one output port. When a new rule with a set DMAC and a new output port is configured, the previous rules are removed from the hardware. Later, if the new configuration is deleted, the previous rules get reinstalled in hardware.
Note that all sent packets from the Router Table are without a VLAN header (untagged).
Supported Router Apply Actions
OFPAT_OUTPUT—the packet is sent out to a port (may be controller port)
OFPAT_DEC_NW_TTL—decrements network TTL
OFPAT_SET_DMAC—OFPAT_SET_FIELD with OFPXMT_OFB_ETH_DST
DROP—drops packet
When an output action is implemented, DEC_TTL and SET_DMAC must also be set.
Supported Router Set Fields
OXM_OF_ETH_DST—sets the destination MAC address of the packet
Supported Router Matching Rules
OXM_OF_IPV4_DST—matches destination IPv4 address
OXM_OF_IPV6_DST—matches destination IPv6 address