OpenFlow 1.3 Workflow

The OpenFlow (OF) pipeline is deployed in parallel to the usual NVIDIA Onyx pipeline.

The ingress port must be deployed in hybrid mode so as to serve both the OF and normal NVIDIA Onyx pipeline.

The ingress packet, which passes the VLAN and Spanning Tree filters and is a match to the user ACL table, either progresses to the regular NVIDIA Onyx flow or the OpenFlow pipeline depending on the port coupling.

The following table presents a general summary of the capabilities of the OpenFlow 1.3 pipeline. They are also described further on in the document.

An Access Control List (ACL) is a list of permissions attached to an object, to filter or match switches packets. When the pattern is matched at the hardware lookup engine, a specified action (e.g. permit/deny) is applied. The rule fields represent flow characteristics such as source and destination addresses, protocol and VLAN ID.

ACL support currently allows actions of permit or deny rules, and supports only ingress direction. ACL search pattern can be taken from either L2 or L3 fields.

Supported ACL Matching Rules

Ingress packets, arriving the ACL, are matched against any combination of the following parameters (defined as the key):

• OXM_OF_METADATA—matches according to metadata

• OXM_OF_IN_PORT—matches according to ingress port (exact match or wildcard)

• OXM_OF_ETH_SRC—matches source MAC address

• OXM_OF_ETH_DST—matches destination MAC address

• OXM_OF_ETH_TYPE—matches EtherType

  Warning

  When match rule is set to match eth_type 9100, VLAN ID matching does not work.

• OXM_OF_VLAN_VID—matches VLAN ID

• OXM_OF_VLAN_PCP—matches priority level

• OXM_OF_IPV4_SRC—matches source IPv4 address

• OXM_OF_IPV4_DST—matches destination IPv4 address

• OXM_OF_IPV6_SRC—matches source IPv6 address

• OXM_OF_IPV6_DST—matches destination IPv6 address

• OXM_OF_IPV6_ND_TARGET

  Warning

  OXM_OF_IPV6_ND_TARGET match rule is not supported.

• OXM_OF_IP_PROTO—matches IP protocols (exact match or wildcard)

• OXM_OF_IP_DSCP—matches IP DSCP field (exact match or wildcard)

• OXM_OF_IP_ECN—matches network ECN (exact match or wildcard)

• OXM_OF_NW_TTL—matches network TTL (exact match or wildcard)

• OXM_OF_TCP_SRC—matches source TCP

• OXM_OF_TCP_DST—matches destination TCP

• OXM_OF_UDP_SRC—matches source UDP

• OXM_OF_UDP_DST—matches destination UDP

• OXM_OF_SCTP_SRC—matches source SCTP

• OXM_OF_SCTP_DST—matches destination SCTP

• OXM_OF_ICMPV4_TYPE—matches ICMP type

• OXM_OF_ICMPV4_CODE—matches ICMP code

• OXM_OF_ARP_OP—matches ARP OP code

• OXM_OF_ARP_SPA—matches sender protocol address

• OXM_OF_ARP_TPA—matches target protocol address

There is a default set of match keys configured. To see what it is, please run the command “show openflow table match-keys” on your machine. To alter it, please use the command “openflow table match-keys”.

Non-standard Matches

OpenFlow 1.3 is able to match non-standard OpenFlow matching rules by mapping them to standard ones. The following non-standard matches are supported:

• Matching source/destination IPv4 address encapsulated with MPLS labels (up to 6 MPLS labels can be skipped)—ip_src_inner/ip_dst_inner is mapped to OXM_OF_IPV4_SRC, OXM_OF_IPV4_DST

• Table configuration:

  Copy

  Copied!

              
  
              
  openflow table 0 match-keys dl_dst dl_src dl_type mpls_label vlan_vid
  openflow table 10 match-keys ignr_eth_type ip_dst_inner ip_src_inner
          

  The ignr_eth_type is needed to ignore the Ethertype of IP that is required by OpenFlow to set to as a prerequisite to match on IP addresses.

• Rules:

  Copy

  Copied!

              
  
              
  openflow add-flows 1 table=0,mpls,mpls_label:32,actions=goto_table=10 
  openflow add-flows 2 table=10,ip,nw_src=10.10.10.0/24,nw_dst=10.10.20.0/24,
  actions=output:127
          

  The above matches IP address from 10.10.10.0/24 to 10.10.20.0/24 which have MPLS label 32 as the first label.

Supported Rule Table Instructions

The intercepted packet is processed according to the instructions on the rule tables. The supported instructions are as follows:
DROP—drops packet

• OFPIT_GOTO_TABLE—sends the packet for processing by another rule table

• OFPIT_METER—policer function; drops packet if it exceeds kbps/pktps limit

• OFPIT_WRITE_METADATA—writes meta-data with mask <METADATA>/0xFFF

• OFPIT_EXPERIMENTE—sends the packet for processing by another controller

• OFPIT_APPLY_ACTIONS—applies certain actions specified in the section below

Supported ACL Apply Actions

The following actions are applied on ingress packets once a match is achieved on the ACL table:

• OFPAT_OUTPUT—the packet is sent out to a port (may also be a controller port)

• OFPAT_GROUP—the packet is sent out to a group
  3 types of group ports are supported:

  • All: The packet is broadcasted on all ports which are part of the defined group

  • Selected: The packets are distributed toward the group ports according to a weight mechanism

  • Fast-Failover (FF): FF is a group of ports, one of which is defined as the primary port through which the packets are transported. In a failure scenario (defined as part of the group definition), traffic becomes transported through the most eligible backup port (from the list of backup ports). Once the failure scenario ends, traffic is routed again through the primary port

• OFPAT_POP_VLAN—strips 802.1Q (VLAN) tag from the packet

• OFPAT_PUSH_VLAN—adds 802.1Q (VLAN) tag from the packet

• OFPAT_SET_NW_TTL—modifies network TTL

• OFPAT_DEC_NW_TTL—decrements network TTL

• OFPAT_SET_FIELD—ACL set fields detailed in section below

• Normal

Supported ACL Set Fields

The following modifications may be implemented on ingress packets:

• OXM_OF_ETH_SRC—sets the source MAC address of the packet

• OXM_OF_ETH_DST—sets the destination MAC address of the packet

• OXM_OF_VLAN_VID—sets the VLAN ID of the packet

• OXM_OF_VLAN_PCP—sets the VLAN priority code point (PCP; 0-7)

• OXM_OF_IP_DSCP—sets IP DSCP

• OXM_OF_IP_ECN—sets network ECN

• NXM_NX_CT_NW_SRC^*—sets the source IP address of the packet

• NXM_NX_CT_NW_DST^*—sets the destination IP address of the packet

• NXM_NX_CT_TP_SRC^*—sets the source L4 port of the packet

• NXM_NX_CT_TP_DST^*—sets the destination L4 port of the packet

*Supported only on Spectrum-2 and Spectrum-3 systems.

Supported ACL Meters

• ACL tables support up to 968 meters with 1 band (drop) per meter.

• Valid meter ID range: 1-969

• Only the rate or the burst size fields can be modified using OFPMC_MODIFY

• OFPMF_BURST meter type can be OFPMF_KBPS (KB/s) or OFPMF_PKTPS (number of packets per second) but not both

• Meter actions:

• OFPMBT_DROP—drops packet according to meter configuration

FDB Table (250)

The FDB table is the same one shared with regular NVIDIA Onyx configuration (e.g., learning, static macs, and so forth). The cumulative number of supported FDB rules is 88KB. FDB may only configure rules with priority of 0x8000. Hard timeout is supported for FDB table rules. FDB rules cannot have wildcard on VID/ETH_DST.

The default action for the FDB table is normal and this cannot be changed by the user.

Supported FDB Apply Actions

• OFPAT_OUTPUT—the packet is sent out to a port (may be controller port)

• DROP—drops packet

• Normal

Supported FDB Matching Rules

• OXM_OF_VLAN_VID—matches VLAN ID

• OXM_OF_ETH_DST—matches destination MAC address

The OpenFlow router table and the regular NVIDIA Onyx router table share the same hardware resources, but are separated logically.

The cumulative number of supported FDB & router rules is 88K. Hard timeout, where the switch removes a rule after a configured timer expires, is supported for router table rules. Switch systems ignore rule priority and configure rules according to masklen in DST IPv4/IPv6 match. A rule with action output must have SET_FIELD with ETH_DST and DEC_NW_TTL. The default action for the router table is DROP.

Set DMAC can be assigned only to one output port. When a new rule with a set DMAC and a new output port is configured, the previous rules are removed from the hardware. Later, if the new configuration is deleted, the previous rules get reinstalled in hardware.
Note that all sent packets from the Router Table are without a VLAN header (untagged).

Supported Router Apply Actions

• OFPAT_OUTPUT—the packet is sent out to a port (may be controller port)

• OFPAT_DEC_NW_TTL—decrements network TTL

• OFPAT_SET_DMAC—OFPAT_SET_FIELD with OFPXMT_OFB_ETH_DST

• DROP—drops packet

Supported Router Set Fields

• OXM_OF_ETH_DST—sets the destination MAC address of the packet

Supported Router Matching Rules

• OXM_OF_IPV4_DST—matches destination IPv4 address

• OXM_OF_IPV6_DST—matches destination IPv6 address