Network Validation for North-South Network#

Once all of the switches are up, verify hardware health and configuration before declaring the North-South network ready for workloads.

Prerequisites and inputs#

Confirm network planning and addressing align with your site and DGX model; refer to Network planning and design and subnet guidance in that chapter where applicable.
Complete BCM bootstrap through head-node and rack-side OOB so switches can be reached for provisioning; follow BCM bootstrap workflow.
Run netautogen and related tooling so BCM holds the expected devices, categories, and addressing; refer to Application workflow and Input file format and manifest.

Fabric underlay routing#

Check the following components for fabric underlay routing:

EVPN Anycast Gateway

Anycast gateway is enabled on all VTEPs.
All VTEPs share:
- Same gateway IP.
- Same virtual MAC (for example, 00:00:5E:00:01:<fabric-id>).
Hosts see a consistent default gateway across all racks.
Traffic is routed locally at the ingress VTEP (symmetric IRB).
ARP/ND information is distributed via EVPN Type-2 routes (MAC/IP).

Fabric overlay routing (EVPN)#

To show the Ethernet segments across all VNIs, run the following command:

To show the Ethernet segments across all VNIs:
```
nv show evpn multihoming
```
Figure 19 Ethernet segments across all VNIs#

Show Ethernet segment information#

The following steps show how to show the Ethernet segments across all VNIs, information about a specific ESI, the Ethernet segments for a specific VNI, the Ethernet segments across all VNIs learned through type-1 and type-4 routes, and verify ESI to VRF information.

To show the Ethernet segments across all VNIs, run the following command:
```
$ nv show evpn multihoming esi
```
Figure 20 Ethernet segments across all VNIs#
To show information about a specific ESI, run the following command:
```
$ nv show evpn multihoming esi <ESI_ID>
```
Figure 21 Ethernet segments across all VNIs#
To show the Ethernet segments for a specific VNI, run the following command:
```
$ nv show evpn vni <VLAN> multihoming esi
```
Figure 22 Ethernet segments across all VNIs#
To show the Ethernet segments across all VNIs learned through type-1 and type-4 routes, run the NVUE command:
```
$ nv show evpn multihoming bgp-info esi
```
Figure 23 Ethernet segments across all VNIs#
To verify ESI to VRF information, run the following command:
```
$ sudo vtysh -c 'show bgp l2vpn evpn es-vrf'
```
Figure 24 Ethernet segments across all VNIs#
To verify EVPN type-1 and type-4 routes learned, run the following command:
```
$ sudo vtysh -c 'show bgp l2vpn evpn route type ead'
```
Figure 25 Ethernet segments across all VNIs#
To verify NVE interface (VTEP) state and configuration, run the following command:
```
$ nv show nve vxlan
```
Figure 26 Ethernet segments across all VNIs#

Customer network reachability#

Check the following components for customer network reachability:

Customer edge to cluster border
- BGP IPv4 AF neighbors
- BGP prefix exchange
- Next-hop reachability
- Packet loss
Customer (external) to cluster (internal)
- End-to-end reachability (if available), dependent on end hosts connected to the fabric (for example, DGX)
- Packet loss

Negative testing#

Check the following components for negative testing:

Customer edge failover
- Disable switch(es) / link(s) / BGP neighbor(s)
- End-to-end reachability
- Packet loss / recovery time
Cluster fabric failover
- Disable switch(es) / link(s) / BGP neighbor(s)
- End-to-end reachability
- Packet loss / recovery time

EVPN multihoming troubleshooting reference#

The following table summarizes common issues, symptoms, root causes, and fixes when validating EVPN multihoming and related fabric behavior.

Issue	Symptoms	Root cause	How to fix
LACP mismatch / one-sided	Only one link forwarding, low bandwidth	LACP mode mismatch (active/passive), config mismatch	Ensure both sides use LACP active, same speed/MTU, same port-channel config
One link down	Loss of redundancy, traffic imbalance	Physical issue, cabling, optics	Check link state, optics, cabling; verify interface is up on both sides
ESI mismatch	EVPN-MH not forming	Different ESI configured on each switch	Configure identical ESI on both switches
Missing ES routes (Type-1)	MH not recognized by peers	EVPN not advertising ES routes	Verify BGP EVPN is enabled and ES routes are advertised
Missing Type-4 routes	DF election not working	Control-plane issue or EVPN config missing	Check BGP EVPN neighbors, ensure Type-4 routes exchanged
DF election failure	Traffic blackhole or duplication	Both switches think DF or no DF	Verify Type-4 routes, check DF state, ensure consistent config
MAC flapping	Logs show MAC moving between switches	Loop, incorrect cabling, or MH misconfig	Fix cabling, verify ESI consistency, check for loops
Split-brain	Both switches act independently	Peer/control-plane failure	Check peer-link or EVPN control plane, restore synchronization
No MAC learning (Type-2 missing)	Host unreachable from remote racks	MAC not advertised via EVPN	Verify EVPN Type-2 routes, check MAC learning on local switch
ARP/ND failure	Cannot resolve IP → MAC	ARP suppression or sync issue	Check ARP table, verify EVPN Type-2 routes
Asymmetric routing	Traffic enters one switch, exits another	Incorrect hashing or routing design	Verify ECMP/hash settings, check routing consistency
Traffic blackholing	Intermittent or total traffic loss	Missing DF, wrong VNI, or routing issue	Check DF election, VNI mapping, and routing tables
VXLAN/VNI mismatch	No connectivity across fabric	Wrong VNI or VLAN mapping	Verify VNI-to-VLAN mapping on all VTEPs
Host hashing imbalance	One link heavily used	LACP hashing behavior	Adjust hashing algorithm (L3/L4), validate traffic distribution
MTU mismatch	Packet drops, performance issues	Different MTU across path	Align MTU across host, leaf, and fabric (VXLAN requires larger MTU)