Ethernet Cluster Networking Validation#

Once all of the switches are up, they need to be verified for hardware health and proper configuration.

Platform Health Check#

Cumulus OS +5.x version only

Check Type

Commands

Overall System Health: Summarizes switch operational state: hardware, software services, configuration status.

nv show system health

Hardware Health: Displays fan speed, PSU status, temperatures, and other platform sensors.

nv show platform
nv show platform temperature
nv show platform fan
nv show platform psu
nv show platform sensor

Service and Process Health: Confirms whether key daemons (BGP, LLDP, switchd, nvlinkd, etc.) are running.

nv show system processes
nv show system services

Interface Status

nv show interface
nv show interface status
nv show interface counters

Find Interface MACs

nv show bridge domain br_default mac

Cabling Quality (L1)#

Cumulus OS +5.x version only

Check Type

Commands

Check Optical Cable / Transceiver Info

nv show interface <ifname> transceiver
nv show interface <ifname> counters
ethtool -t <ifname>
ethtool <ifname>

Fabric Underlay Routing#

  1. Fabric Overlay Routing (EVPN)

    nv show evpn multihoming
    
    EVPN Multihoming

    Figure 6 EVPN Multihoming (command-line output)#

  2. Show Ethernet Segment Information

    To show the Ethernet segments across all VNIs:

    nv show evpn multihoming esi
    
    EVPN Multihoming ESI

    Figure 7 EVPN Multihoming ESI#

  3. To show information about a specific ESI:

    nv show evpn multihoming esi <ESI ID>
    
    EVPN Multihoming a specific ESI

    Figure 8 EVPN Multihoming a specific ESI#

  4. To show the Ethernet segments for a specific VNI:

    nv show evpn vni <vlan> multihoming esi
    
    EVPN Multihoming a specific VNI

    Figure 9 EVPN Multihoming a specific VNI#

  5. To show the Ethernet segments across all VNIs learned through type-1 and type-4 routes, run the NVUE:

    nv show evpn multihoming bgp-info esi
    
    EVPN Multihoming BGP Info ESI

    Figure 10 EVPN Multihoming BGP Info ESI#

  6. Verify ESI to VRF information:

    sudo vtysh -c 'show bgp l2vpn evpn es-vrf'
    
    Show BGP EVPN ES-VRF

    Figure 11 Show BGP EVPN ES-VRF (command-line output)#

  7. Verify EVPN type-1 and type-4 routes learned:

    nv show evpn multihoming bgp-info esi
    
    EVPN Routes Multihoming BGP Info ESI

    Figure 12 EVPN Routes Multihoming BGP Info ESI#

  8. Verify NVE interface (VTEP) state and configuration:

    nv show nve vxlan
    
    NVE VXLAN

    Figure 13 NVE VXLAN (command-line output)#

Customer Network Reachability#

  • Customer Edge to Cluster Border

  • BGP IPv4 AF Neighbors

  • BGP Prefix Exchange

  • Next-Hop Reachability

  • Packet Loss

  • Customer (external) to Cluster (internal)

  • End-to-End Reachability (if available)

  • Dependent on end-host connected to fabric (ex: DGX)

  • Packet Loss

Negative Testing#

Customer Edge Failover

  • Disable Switch(es) / Link(s) / BGP Neighbor(s)

  • End-to-End Reachability

  • Packet Loss / Recovery Time

Cluster Fabric Failover

  • Disable Switch(es) / Link(s) / BGP Neighbor(s)

  • End-to-End Reachability

  • Packet Loss / Recovery Time