Illegal Drop Counters

Illegal drop counters provide better packet drop visibility in SONiC. They are a mechanism for counting packet drops that occur due to different reasons. Counters might drop based on one of the following scenarios:

  • drop reason – a condition which led to switch pipeline “drop” decision
  • drop reason family – group of drop reasons belonging to a pipeline stage (L2, L3, TUNNEL, ACL)
  • drop counter – a user-created counter which can be associated with one or several drop reason
  • drop counter type – ingress/egress direction for switch/port level drops

Illegal drop counters are included in the base SONiC image, so there is nothing to install.

Supported Drop Reasons

THe layer 2 drop family includes the following drop reasons:

Drop ReasonDescription
SMAC_MULTICASTSource MAC is multicast
SMAC_EQUALS_DMACSource MAC equals destination MAC
DMAC_RESERVEDDestination MAC is reserved (destination MAC=01-80-C2-00-00-0x)
VLAN_TAG_NOT_ALLOWEDVLAN tag not allowed (frame is tagged when port is dropping tagged, or is untagged when dropping untagged)
INGRESS_VLAN_FILTERIngress VLAN filter (incoming frames are tagged with a VLAN that is not configured on the ingress bridge port)
INGRESS_STP_FILTERIngress STP filter
FDB_UC_DISCARDUnicast FDB table action discard
FDB_MC_DISCARDMulticast FDB table empty TX list
L2_LOOPBACK_FILTERPort L2 loopback filter (packet egressing and ingressing on the same port and VLAN)
L2_ANYAny L2 pipeline drop (all L2 drop reasons together)

The layer 3 family includes the following drop reasons:

Drop ReasonDescription
EXCEEDS_L3_MTUPacket size is larger than the L3 (router interface) MTU
TTLTTL expired
L3_LOOPBACK_FILTERRIF L3 loopback filter (packet ingressing and egressing on the same RIF)
NON_ROUTABLENon-routable packet (IGMP v1 v2 v3 membership query, IGMP v1 membership report, IGMP v2 membership report, IGMP v2 leave group, IGMP v3 membership report)
NO_L3_HEADERDestination MAC is the router MAC; however, packet is not routable (isn’t IP or MPLS)
IP_HEADER_ERRORIP header error (due to header checksum, bad IP version or IPv4 IHL too short)
UC_DIP_MC_DMACUnicast destination IP with non-unicast (multicast or broadcast) destination MAC
DIP_LOOPBACKDestination IP is loopback address (for IPv4: destination IP=127.0.0.0/8, for IPv6: destination IP=::1/128 or destination IP=0:0:0:0:0:ffff:7f00:0/104)
SIP_LOOPBACKSource IP is loopback address (for IPv4: source IP=127.0.0.0/8, for IPv6: source IP=::1/128)
SIP_MCSource IP is multicast address (for IPv4: source IP=224.0.0.0/4, for IPv6: source IP=FF00::/8)
SIP_CLASS_ESource IP is in class E (IPv4 AND Source IP=240.0.0.0/4 AND Source IP!=255.255.255.255)
SIP_UNSPECIFIEDSource IP unspecified (for IPv4: source IP=0.0.0.0/32, for IPv6: source IP=::0)
MC_DMAC_MISMATCHDestination IP is multicast but destination MAC isn’t (destination IP is multicast and for IPv4: destination MAC!={01-00-5E-0 (25 bits), dip[22:0]}, for IPv6: destination MAC!={33-33, DIP[31:0]})
SIP_EQUALS_DIPSource IP equals destination IP
SIP_BCIPv4 source IP is limited broadcast (source IP=255.255.255.255)
DIP_LOCALIPv4 destination IP is local network (destination IP=0.0.0.0/8)
DIP_LINK_LOCALIPv4 unicast destination IP is link local (destination IP=169.254.0.0/16)
SIP_LINK_LOCALIPv4 source IP is link local (source IP=169.254.0.0/16)
IPV6_MC_SCOPE0IPv6 destination in multicast scope 0 reserved (destination IP=ff:x0:/16)
IPV6_MC_SCOPE1IPv6 destination in multicast scope 1 interface-local (destination IP=ff:x1:/16)
IRIF_DISABLEDIngress RIF is disabled
ERIF_DISABLEDEgress RIF is disabled
LPM4_MISSIPv4 routing table (LPM) unicast miss
LPM6_MISSIPv6 routing table (LPM) unicast miss
BLACKHOLE_ROUTEBlack hole route (discard by route entry)
BLACKHOLE_ARPBlack hole ARP/neighbor (discard by ARP or neighbor entries)
UNRESOLVED_NEXT_HOPUnresolved next hop (missing ARP entry)
L3_ANYAny L3 pipeline drop (all L3 drop reasons together)

The tunnel drop family includes:

Drop ReasonDescription
DECAP_ERRORPacket decapsulation failed (need to decapsulate too many bytes, remaining headers are too short)

The ACL drop family includes:

Drop ReasonDescription
ACL_ANYPacket is dropped due to configured ACL rules, all stages/bind points combinations

Configuration Commands

config dropcounters installDescription
config dropcounters install [OPTIONS] COUNTER_NAME COUNTER_TYPE REASONSInstalls a new drop counter.
Example
admin@switch:~$ sudo config dropcounters install COUNTER_1 SWITCH_INGRESS_DROPS L2_ANY
config dropcounters add-reasonsDescription
config dropcounters add-reasons [OPTIONS] COUNTER_NAME REASONSAdds reasons to an existing drop counter.
Example
admin@switch:~$ sudo config dropcounters add-reasons COUNTER_1 L3_ANY
config dropcounters remove-reasonsDescription
config dropcounters remove-reasons [OPTIONS] COUNTER_NAME REASONSRemove reasons from an existing drop counter
Example
admin@switch:~$ sudo config dropcounters remove-reasons COUNTER_1 L2_ANY
config dropcounters deleteDescription
config dropcounters delete [OPTIONS] COUNTER_NAMEDeletes an existing drop counter
Example
admin@switch:~$ sudo config dropcounters delete COUNTER_1
show dropcounters capabilitiesDescription
show dropcounters capabilitiesDisplays a total allowed number of drop counters and a list of the supported drop reasons for a particular counter type.
Example
admin@switch:~$ show dropcounters capabilities
Counter Type Total
——————– ——-
SWITCH_EGRESS_DROPS 31
SWITCH_INGRESS_DROPS 31

SWITCH_INGRESS_DROPS
ACL_ANY
DECAP_ERROR
BLACKHOLE_ARP
LPM6_MISS
ERIF_DISABLED
IPV6_MC_SCOPE1
IPV6_MC_SCOPE0
SIP_LINK_LOCAL
DIP_LINK_LOCAL
DIP_LOCAL
MC_DMAC_MISMATCH
FDB_MC_DISCARD
L3_ANY
INGRESS_VLAN_FILTER
LPM4_MISS
L2_ANY
IRIF_DISABLED
SMAC_MULTICAST
FDB_UC_DISCARD
DMAC_RESERVED
INGRESS_STP_FILTER
DIP_LOOPBACK
L2_LOOPBACK_FILTER
VLAN_TAG_NOT_ALLOWED
SIP_LOOPBACK
UNRESOLVED_NEXT_HOP
TTL
SIP_BC
SMAC_EQUALS_DMAC
UC_DIP_MC_DMAC
EXCEEDS_L3_MTU
L3_LOOPBACK_FILTER
SIP_UNSPECIFIED
NON_ROUTABLE
BLACKHOLE_ROUTE
NO_L3_HEADER
IP_HEADER_ERROR
SIP_MC
SIP_CLASS_E
show dropcounters configurationDescription
show dropcounters configurationShows the current drop counter configuration.
Example
admin@switch:~$ show dropcounters configuration
Counter Alias Group Type Reasons Description
——— ——— ——- ——————– ——— ————-
COUNTER_1 COUNTER_1 N/A SWITCH_INGRESS_DROPS L2_ANY N/A
L3_ANY
show dropcounters countsDescription
show dropcounters countsShows drop counts.
Example
admin@switch:~$ show dropcounters counts
IFACE STATE RX_ERR RX_DROPS TX_ERR TX_DROPS
———– ——- ——– ———- ——– ———-
Ethernet0 U 80004 0 0 0
Ethernet1 D 0 0 0 0
Ethernet2 D 0 0 0 0
Ethernet3 D 0 0 0 0
……………………

DEVICE COUNTER_1
—————– ———–
r-qa-sw-eth-20123 0

Limitations

  • Illegal drop counters and What Just Happened counters are mutually exclusive; they cannot be used simultaneously.
  • The total available amount of drop counters is actually lower than what is output from show dropcounters capabilities due to a complex internal logic.
  • An individual drop reason can be used only once in a configuration; multiple counters cannot share the same drop reason.
  • Individual L2_ANY (including all L2 drop reasons) and L3_ANY (including all L3 drop reasons) reasons can be used only once in a configuration.
  • All NVIDIA Spectrum-based switches support the same set of drop reasons, but later Spectrum models support more drop counters (Spectrum 1-based systems support fewer drop reasons than NVIDIA Spectrum 2-based systems, for example).
  • All Spectrum-based switches support only one counter type: SWITCH_INGRESS_DROPS (switch level, ingress direction).
  • The following drop reasons are disabled. Packets matching the following drop reasons are forwarded, not dropped:
    • SIP_EQUAL_DIP (source ip = destination ip)
    • L3_LOOPBACK_FILTER (packet egressing on same RIF as ingressing)
  • The following drop reasons require specific system configuration via developers debug utilities:
    • IRIF_DISABLED (ingress RIF is disabled)
    • ERIF_DISABLED (egress RIF is disabled)
    • BLACKHOLE_ARP (discard by ARP)
  • The following drop reasons are never matched because of internal default blackhole routes for 0.0.0.0/0 and ::/0:
    • LPM4_MISS (IPv4 routing table unicast miss)
    • LPM6_MISS (IPv6 routing table unicast miss)
  • The following drop reason is never matched because such traffic is trapped at the CPU:
    • UNRESOLVED_NEXT_HOP (missing ARP entries)