MetroX XC Routing Overview

NVIDIA Docs Hub NVIDIA Networking Networking Software Switch Software NVIDIA NVDA-OS XC User Manual for NVIDIA MetroX-3 XC Appliance v18.02.2200 LTS MetroX XC Routing Overview

InfiniBand Interfaces

show interfaces ib

	show interfaces ib <interface> Displays information on InfiniBand interfaces.
Syntax Description	Interface	Device/port (e.g., 1/1)
Default	N/A
Configuration Mode	Any command mode
History	18.01.5000
History	18.02.1000
Example	metrox (config) # show interfaces ib 1/1 IB1/1: Admin state : Enabled Logical port state : LinkDown Physical port state: Polling GUID : e8eb:d303:0066:55fe Supported speeds : sdr fdr edr Speed : N/A Supported widths : 1x 2x 4x Max supported MTU : 4096 MTU : 4096 VL capabilities : VL0-VL3 Description : IB Subnet : InfiniBand-default Phy-profile : high-speed-ber Rx: packets : 0 bytes : 0 error packets: 0 Tx: packets : 0 bytes : 0 error packets: 0
Related Commands	show interfaces long-haul
Notes	If the interface is not specified, then the command prints all the InfiniBand interfaces.

show interfaces ib brief

	show interfaces ib brief Displays brief information on the InfiniBand interfaces.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	18.01.5000
History	18.02.1000
Example	metrox (config) # show interfaces ib brief ------------------------------------------------------------- Name Admin state Logical Port state Speed ------------------------------------------------------------- IB1/1 Enabled LinkActive edr IB2/1 Enabled LinkActive edr
Related Commands	show interfaces ib
Notes

interface ib shutdown

	interface ib <interface> shutdown no interface ib <interface> shutdown Disables/Enables InfiniBand physical interface.
Syntax Description	Interface	Device/port (e.g., 1/1)
Default	N/A
Configuration Mode	config
History	18.01.5000
Example	metrox (config) # interface ib 1/1 shutdown
Related Commands	show interfaces ib
Notes

interface ib description

	interface ib <interface> description no interface ib <interface> description Adds a description for an interface. The no form of the command removes a description for an interface.
Syntax Description	Interface	Device/port (e.g., 1/1)
Syntax Description	Description	A free-form string that has no semantics other than being displayed when the interface records are listed
Default	N/A
Configuration Mode	config
History	18.01.5000
Example	metrox (config) # interface ib 1/1 description first port
Related Commands	show interfaces ib
Notes

Long-Haul Interfaces

show interfaces long-haul

	show interfaces long-haul <interface> Displays information on long-haul interfaces. If the interface is not specified, then the command prints all the InfiniBand interfaces.
Syntax Description	Interface	Device/port (e.g., 1/1)
Default	N/A
Configuration Mode	Any command mode
History	18.01.5000
Example	metrox (config) # show interfaces long-haul 1/2 LH1/2: Admin state : Enabled Operational state : Up Description : MAC address : E8:EB:D3:66:55:FF MTU : 9216 FEC : auto Supported speeds : 100G Actual speed : Unknown Auto-negotiation : Enabled Actual latency (ms) : 0 Last latency measurement: N/A Remote KA status : Failure Last success KA message : N/A IPv4 address: N/A Remote IPv4 address: N/A Rx: packets : 299 bytes : 20830 error packets: 0 Tx: packets : 14 bytes : 1116 error packets: 0
Related Commands	show interfaces ib
Notes

show interfaces long-haul brief

	show interfaces long-haul brief Displays brief information on the long-haul interfaces.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	18.01.5000
Example	metrox (config) # show interfaces long-haul brief ------------------------------------------------------------- Name Admin state Oper state Speed ------------------------------------------------------------- LH1/2 Enabled Up 100G LH2/2 Enabled Up 100G
Related Commands	show interfaces long-haul
Notes

interface long-haul shutdown

New command	interface long-haul <interface> shutdown no interface long-haul <interface> shutdown Disables/Enables long-haul interface.
Syntax Description	Interface	Device/port (e.g., 1/2)
Default	N/A
Configuration Mode	config
History	18.01.5000
Example	metrox (config) # interface long-haul 1/2 shutdown
Related Commands	show interfaces long-haul
Notes

interface long-haul description

	interface long-haul <interface> description no interface long-haul <interface> description Adds a description for an interface. The no form of the command removes a description for an interface.
Syntax Description	Interface	Device/port (e.g., 1/2)
Syntax Description	Description	A free-form string that has no semantics other than being displayed when the interface records are listed
Default	N/A
Configuration Mode	config
History	18.01.5000
Example	metrox (config) # interface long-haul 1/2 description first port
Related Commands	show interfaces long-haul interface long-haul remote ip address
Notes	Both the local and remote IP addresses should be configured in order to pass traffic.

interface long-haul remote ip address

	interface long-haul <interface> remote ip address <ip-address> no interface long-haul <interface> remote ip address <ip-address> [<ip address>] Enters user-defined remote IPv4 address for the interface. The no form of the command removes the specified remote IPv4 address. If no address is specified, then all IPv4 addresses of this interface are removed.
Syntax Description	interface	Device/port (e.g., 1/2)
Syntax Description	ip-address	IPv4 address
Default	N/A
Configuration Mode	config
History	18.01.5000
Example	metrox (config) # interface long-haul 1/2 remote ip address 1.1.1.1/24
Related Commands	show interfaces long-haul interface long-haul ip address
Notes

interface long-haul ip address

	interface long-haul <interface> ip address <ip-address> <mask> no interface long-haul <interface> ip address <ip-address> [<ip address>] Enters user-defined IPv4 address for the interface. The no form of the command removes the specified IPv4 address. If no address is specified, then all IPv4 addresses of this interface are removed.
Syntax Description	interface	Device/port (e.g., 1/2)
	ip-address	IPv4 address
	mask	There are two possible ways to the mask:/length (i.e. /24) Network address (i.e. 255.255.255.0) The mask length may be configured without a space (i.e. <ipv4-address>/<length>)
Default	N/A
Configuration Mode	config
History	18.01.5000
Example	metrox (config) # interface long-haul 1/2 ip address 1.1.1.1/24
Related Commands	show interfaces long-haul interface long-haul ip address
Notes	Both the local and remote IP addresses should be configured in order to pass traffic.

Routing and ECMP

ip route

	ip route <ip-prefix> <netmask> {<next -hop-ip-address> \| ethernet <port>} [<distance>] no ip route <ip-prefix> <netmask> [<next -hop-ip-address>] Configures a static route. The no form of the command removes the static route configured.
Syntax Description	ip-prefix	IP address.
	netmask	There are two possible ways to input the mask: /<length> (e.g., /24) Network address (e.g., 255.255.255.0)
	next-hop-ip-address	IP address of the next hop
	Default	N/A
Configuration Mode	config
History	8.0.0100
Example	metrox (config) # ip route 80.80.80.0 /24 20.20.20.2
Related Commands	show ip route
Notes	The route is configurable between the Ethernet and InfiniBand subnets

show ip route

	show ip route [static] Displays routing table.
Syntax Description	static	Displays entries added through CLI commands.
Configuration Mode	config
History	18.01.5002
Example	metrox (config) # show ip route Flags: F: Failed to install in H/W B: BFD protected (static route) i: BFD session initializing (static route) x: protecting BFD session failed (static route) c: consistent hashing p: partial programming in H/W VRF Name default: ----------------------------------------------------------------------------------- Destination Mask Flag Gateway Interface Source AD/M ----------------------------------------------------------------------------------- default 0.0.0.0 10.12.67.126 mgmt0 DHCP 1/1 10.12.67.0 255.255.255.128 0.0.0.0 mgmt0 direct 0/0 192.168.2.0 255.255.255.0 c 0.0.0.0 vlan1 direct 0/0
Related Commands	ip route
Notes	If no default route exists, then the message “Route not found” is printed Route next hop is BFD controlled, status is viewable when <all> is inserted in the command, and it will be shown as follows: If route is removed from routing decision it will be marked as “Active” Protected next hops are marked with “B” BFD protected failed/non active neighbors are marked with “BF”

IP Diagnostic Tools

ping

	ping <interface> [-LRUbdfnqrvVaA] [-c count] [-i interval] [-w deadline] [-p pattern] [-s packetsize] [-t ttl] [-I interface or address] [-M mtu discovery hint] [-S sndbuf] [-T timestamp option ] [-Q tos ] [hop1 ...] destination Sends ICMP echo requests to a specified host.
Syntax Description	Interface	Device/port (e.g., 1/2)
Syntax Description	Linux Ping options
Default	N/A
Configuration Mode	config
History	18.01.5002
Example	metrox (config) # ping interface long-haul 1/2 1.1.1.1 PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. 64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=0.456 ms 64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.222 ms ^C --- 1.1.1.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1038ms rtt min/avg/max/mdev = 0.222/0.339/0.456/0.117 ms
Related Commands	traceroute
Notes	When using -I option use the interface name + interface number, for example “ping -I vlan10”

traceroute

	traceroute <interface> [-46dFITUnrAV] [-f first_ttl] [-g gate,...] [-i device] [-m max_ttl] [-N squeries] [-p port] [-t tos] [-l flow_label] [-w waittime] [-q nqueries] [-s src_addr] [-z sendwait] host [packetlen] Traces the route packets take to a destination.
Syntax Description	Interface	Device/port (e.g., 1/2)
	-4	Uses IPv4
	-d	Enables socket level debugging
	-F	Sets DF (“do not fragment” bit) on
	-I	Uses ICMP ECHO for tracerouting
	-T	Uses TCP SYN for tracerouting
	-U	Uses UDP datagram (default) for tracerouting
	-n	Does not resolve IP addresses to their domain names
	-r	Bypasses the normal routing and send directly to a host on an attached network
	-A	Performs AS path lookups in routing registries and print results directly after the corresponding addresses
	-V	Prints version info and exit
	-f	Starts from the first_ttl hop (instead from 1)
	-g	Routes packets throw the specified gateway (maximum 8 for IPv4 and 127 for IPv6)
	-i	Specifies a network interface to operate with
	-m	Sets the max number of hops (max TTL to be reached) Default: 30
	-N	Sets the number of probes to be tried simultaneously Default: 16
	-p	Uses destination port. It is an initial value for the UDP destination port (incremented by each probe, default is 33434), for the ICMP seq number (incremented as well, default from 1), and the constant destination port for TCP tries (default is 80).
	-t	Sets the TOS (IPv4 type of service) or TC (IPv6 traffic class) value for outgoing packets
	-l	Uses specified flow_label for IPv6 packets
	-w	Sets the number of seconds to wait for response to a probe (default is 5.0). Non-integer (float point) values allowed too.
	-q	Sets the number of probes per each hop Default: 3
	-s	Uses source src_addr for outgoing packets
	-z	Sets minimal time interval between probes (default is 0). If the value is more than 10, then it specifies a number in milliseconds, else it is a number of seconds (float point values allowed too).
Default	N/A
Configuration Mode	config
History	18.01.5002
Example	metrox (config) # traceroute interface long-haul 1/2 192.168.10.70 traceroute to 192.168.10.70 (192.168.10.70), 30 hops max, 40 byte packets 1 172.30.0.1 (172.30.0.1) 3.632 ms 2.849 ms 3.544 ms 2 10.222.128.46 (10.222.128.46) 3.176 ms 3.289 ms 3.656 ms 3 10.158.128.30 (10.158.128.30) 15.331 ms 15.819 ms 16.388 ms 4 10.158.128.65 (10.158.128.65) 20.468 ms 7.893 ms 12.27 ms 5 10.7.34.115 (10.7.34.115) 16.405 ms 11.985 ms 12.264 ms 6 192.168.10.70 (192.168.10.70) 16.377 ms 16.091 ms 20.475 ms
Related Commands
Notes	The following flags are not supported: -6, -l, -A When using -i option use the interface name + interface number, for example “traceroute -i mgmt0”

tcpdump

	tcpdump <interface> [-aAdeflLnNOpqRStuUvxX] [-c count] [-C file_size] [-E algo:secret] [-F file] [-i interface] [-M secret] [-r file] [-s snaplen] [-T type] [-w file] [-W filecount] [-y datalinktype] [-Z user] [expression] Invokes standard binary, passing command line parameters straight through. Runs in foreground, printing packets as they arrive, until the user hits Ctrl+C.
Syntax Description	Interface	Device/port (e.g., 1/2)
Syntax Description	Linux Ping options
Default	N/A
Configuration Mode	config
History	18.01.5002
Example
metrox (config) # tcpdump interface long-haul 1/2 ...... 09:37:38.678812 IP 192.168.10.7.ssh > 192.168.10.1.54155: P 1494624:1494800(176) ack 625 win 90 <nop,nop,timestamp 5842763 858672398> 09:37:38.678860 IP 192.168.10.7.ssh > 192.168.10.1.54155: P 1494800:1495104(304) ack 625 win 90 <nop,nop,timestamp 5842763 858672398> ... 9141 packets captured 9142 packets received by filter 0 packets dropped by kernel
Related Commands
Notes	When using -i option use the interface name + interface number, for example “tcpdump -i mgmt0” For all flag options of this command refer to the linux ‘man page’ of tcp dump

System Commands

show system state

	show system state Displays information on the state of the ConnectX cards (adapter cards) of the system.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	18.01.5002
Example	metrox (config) # show system state System state : system-ready System state description: system is ready to send/receive traffic ------------------------------------------------------------------------ Card ID Ports Card type Card state ------------------------------------------------------------------------ 1 IB 1/1, LH 1/2 ConnectX7 card-ready 2 IB 2/1, LH 2/2 ConnectX7 card-ready IB-MGMT IB-MGMT1 ConnectX7 card-ready
Related Commands
Notes

show asic-version

	show asic-version Displays FW versions of the ConnectX7 cards installed on the system
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	18.01.5002
Example	metrox (config) # show asic-version ----------------------------------------------------- Device Type Version ----------------------------------------------------- 1 ConnectX7 28.0.99.1290 2 ConnectX7 28.0.99.1290 IB-MGMT ConnectX7 28.0.99.1290
Related Commands
Notes

MetroX XC Routing Overview

On This Page