Validate Ethernet Cluster Networking

Once all of the switches are up, you need to verify the hardware health and proper configuration.

Platform Health

Check the following components for platform health:

• System

• Internal Components

• Software Processes

• Transceiver/Cable

Cabling Quality (L1)

Check the following components for cabling quality:

• Tx/Rx Signal Level

• Peer Connectivity

• Tx/Rx Signal Integrity

Fabric Underlay Routing

Check the following components for fabric underlay routing:

VRR and EVPN Anycast Gateway

• VRR enabled on ALL VTEPs (same Fabric ID) — This creates a full EVPN Anycast Gateway across the entire fabric.

• All VTEPs use the same gateway MAC — 00:00:5E:00:01:<fabric-id>.

• Hosts in any rack see a consistent L2/L3 boundary.

• ARP/ND is synchronized via EVPN Type-2 routes.

• VTEPs advertise: same MAC, same IP, same fabric-id (EVPN ES-I).

OOB VRR

OOB will have VRR configured, with a unique SVI IP per switch; the gateway will be the same for a single VLAN.

Fabric Overlay Routing (EVPN)

To show the Ethernet segments across all VNIs, run the following command:

1. To show the Ethernet segments across all VNIs:

   nv show evpn multihoming
   

   

   Figure 6 Ethernet Segments Across All VNIs

Show Ethernet Segment Information

The following steps show how to show the Ethernet segments across all VNIs, information about a specific ESI, the Ethernet segments for a specific VNI, the Ethernet segments across all VNIs learned through type-1 and type-4 routes, and verify ESI to VRF information.

1. To show the Ethernet segments across all VNIs, run the following command:

   $ nv show evpn multihoming esi
   

   

   Figure 7 Ethernet Segments Across All VNIs

2. To show information about a specific ESI, run the following command:

   $ nv show evpn multihoming esi <ESI_ID>
   

   

   Figure 8 Ethernet Segments Across All VNIs

4. To show the Ethernet segments for a specific VNI, run the following command:

   $ nv show evpn vni <VLAN> multihoming esi
   

   

   Figure 9 Ethernet Segments Across All VNIs

5. To show the Ethernet segments across all VNIs learned through type-1 and type-4 routes, run the NVUE command:

   $ nv show evpn multihoming bgp-info esi
   

   

   Figure 10 Ethernet Segments Across All VNIs

6. To verify ESI to VRF information, run the following command:

   $ sudo vtysh -c 'show bgp l2vpn evpn es-vrf'
   

   

   Figure 11 Ethernet Segments Across All VNIs

7. To verify EVPN type-1 and type-4 routes learned, run the following command:

   $ sudo vtysh -c 'show bgp l2vpn evpn route type ead'
   

   

   Figure 12 Ethernet Segments Across All VNIs

8. To verify NVE interface (VTEP) state and configuration, run the following command:

   $ nv show nve vxlan
   

   

   Figure 13 Ethernet Segments Across All VNIs

Customer Network Reachability

Check the following components for customer network reachability:

1. Customer Edge to Cluster Border

   • BGP IPv4 AF Neighbors

   • BGP Prefix Exchange

   • Next-Hop Reachability

   • Packet Loss

2. Customer (external) to Cluster (internal)

   • End-to-End Reachability (if available) - Dependent on end-host connected to fabric (ex: DGX)

   • Packet Loss

Negative Testing

Check the following components for negative testing:

• Customer Edge Failover

  • Disable Switch(es) / Link(s) / BGP Neighbor(s)

  • End-to-End Reachability

  • Packet Loss / Recovery Time

• Cluster Fabric Failover

  • Disable Switch(es) / Link(s) / BGP Neighbor(s)

  • End-to-End Reachability

  • Packet Loss / Recovery Time