MetroX XC Routing Overview

NVIDIA NVDA-OS XC User Manual for NVIDIA MetroX-3 XC Appliance v18.02.1000

show interfaces ib

show interfaces ib <interface>

Displays information on InfiniBand interfaces.

Syntax Description

Interface

Device/port (e.g., 1/1)

Default

N/A

Configuration Mode

Any command mode

History

18.01.5000

18.02.1000

Example

metrox (config) # show interfaces ib 1/1

IB1/1:
Admin state : Enabled
Logical port state : LinkDown
Physical port state: Polling
GUID : e8eb:d303:0066:55fe
Supported speeds : sdr fdr edr
Speed : N/A
Supported widths : 1x 2x 4x
Max supported MTU : 4096
MTU : 4096
VL capabilities : VL0-VL3
Description :
IB Subnet : InfiniBand-default
Phy-profile : high-speed-ber

Rx:
packets : 0
bytes : 0
error packets: 0

Tx:
packets : 0
bytes : 0
error packets: 0

Related Commands

show interfaces long-haul

Notes

If the interface is not specified, then the command prints all the InfiniBand interfaces.

show interfaces ib brief

show interfaces ib brief

Displays brief information on the InfiniBand interfaces.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

18.01.5000

18.02.1000

Example

metrox (config) # show interfaces ib brief 
-------------------------------------------------------------
Name Admin state Logical Port state Speed
-------------------------------------------------------------
IB1/1 Enabled LinkActive edr
IB2/1 Enabled LinkActive edr

Related Commands

show interfaces ib

Notes

interface ib shutdown

interface ib <interface> shutdown
no interface ib <interface> shutdown

Disables/Enables InfiniBand physical interface.

Syntax Description

Interface

Device/port (e.g., 1/1)

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface ib 1/1 shutdown

Related Commands

show interfaces ib

Notes

interface ib description

interface ib <interface> description
no interface ib <interface> description

Adds a description for an interface.
The no form of the command removes a description for an interface.

Syntax Description

Interface

Device/port (e.g., 1/1)

Description

A free-form string that has no semantics other than being displayed when the interface records are listed

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface ib 1/1 description first port

Related Commands

show interfaces ib

Notes

show interfaces long-haul

show interfaces long-haul <interface>

Displays information on long-haul interfaces.
If the interface is not specified, then the command prints all the InfiniBand
interfaces.

Syntax Description

Interface

Device/port (e.g., 1/1)

Default

N/A

Configuration Mode

Any command mode

History

18.01.5000

Example

metrox (config) # show interfaces long-haul 1/2

LH1/2:
Admin state : Enabled
Operational state : Up
Description :
MAC address : E8:EB:D3:66:55:FF
MTU : 9216
FEC : auto
Supported speeds : 100G
Actual speed : Unknown
Auto-negotiation : Enabled
Actual latency (ms) : 0
Last latency measurement: N/A
Remote KA status : Failure
Last success KA message : N/A

IPv4 address:
N/A

Remote IPv4 address:
N/A

Rx:
packets : 299
bytes : 20830
error packets: 0

Tx:
packets : 14
bytes : 1116
error packets: 0

Related Commands

show interfaces ib

Notes

show interfaces long-haul brief

show interfaces long-haul brief

Displays brief information on the long-haul interfaces.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

18.01.5000

Example

metrox (config) # show interfaces long-haul brief

-------------------------------------------------------------
Name Admin state Oper state Speed
-------------------------------------------------------------
LH1/2 Enabled Up 100G
LH2/2 Enabled Up 100G

Related Commands

show interfaces long-haul

Notes

interface long-haul shutdown

New command

interface long-haul <interface> shutdown
no interface long-haul <interface> shutdown

Disables/Enables long-haul interface.

Syntax Description

Interface

Device/port (e.g., 1/2)

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface long-haul 1/2 shutdown

Related Commands

show interfaces long-haul

Notes

interface long-haul description

interface long-haul <interface> description
no interface long-haul <interface> description

Adds a description for an interface.
The no form of the command removes a description for an interface.

Syntax Description

Interface

Device/port (e.g., 1/2)

Description

A free-form string that has no semantics other than being displayed
when the interface records are listed

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface long-haul 1/2 description first port

Related Commands

show interfaces long-haul
interface long-haul remote ip address

Notes

Both the local and remote IP addresses should be configured in order to pass traffic.

interface long-haul remote ip address

interface long-haul <interface> remote ip address <ip-address>
no interface long-haul <interface> remote ip address <ip-address> [<ip address>]

Enters user-defined remote IPv4 address for the interface.
The no form of the command removes the specified remote IPv4 address. If no address is
specified, then all IPv4 addresses of this interface are removed.

Syntax Description

interface

Device/port (e.g., 1/2)

ip-address

IPv4 address

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface long-haul 1/2 remote ip address 1.1.1.1/24

Related Commands

show interfaces long-haul
interface long-haul ip address

Notes

interface long-haul ip address

interface long-haul <interface> ip address <ip-address> <mask>
no interface long-haul <interface> ip address <ip-address> [<ip address>]

Enters user-defined IPv4 address for the interface.
The no form of the command removes the specified IPv4 address. If no address is
specified, then all IPv4 addresses of this interface are removed.

Syntax Description

interface

Device/port (e.g., 1/2)

ip-address

IPv4 address

mask

There are two possible ways to the mask:/length (i.e. /24)
Network address (i.e. 255.255.255.0) The mask length may be configured without a space (i.e. <ipv4-address>/<length>)

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface long-haul 1/2 ip address 1.1.1.1/24

Related Commands

show interfaces long-haul
interface long-haul ip address

Notes

Both the local and remote IP addresses should be configured in order to pass traffic.

ip route

ip route <ip-prefix> <netmask> {<next -hop-ip-address> | ethernet <port>} [<distance>]
no ip route <ip-prefix> <netmask> [<next -hop-ip-address>]

Configures a static route.
The no form of the command removes the static route configured.

Syntax Description

ip-prefix

IP address.

netmask

There are two possible ways to input the mask:

  • /<length> (e.g., /24)

  • Network address (e.g., 255.255.255.0)

next-hop-ip-address

IP address of the next hop

Default

N/A

Configuration Mode

config

History

8.0.0100

Example

    
metrox (config) # ip route 80.80.80.0 /24 20.20.20.2    

Related Commands

show ip route

Notes

The route is configurable between the Ethernet and InfiniBand subnets

show ip route

show ip route [static]

Displays routing table.

Syntax Description

static

Displays entries added through CLI commands.

Configuration Mode

config

History

18.01.5002

Example

metrox (config) # show ip route

Flags:
F: Failed to install in H/W
B: BFD protected (static route)
i: BFD session initializing (static route)
x: protecting BFD session failed (static route)
c: consistent hashing
p: partial programming in H/W

VRF Name default:
-----------------------------------------------------------------------------------
Destination Mask Flag Gateway Interface Source AD/M
-----------------------------------------------------------------------------------
default 0.0.0.0 10.12.67.126 mgmt0 DHCP 1/1
10.12.67.0 255.255.255.128 0.0.0.0 mgmt0 direct 0/0
192.168.2.0 255.255.255.0 c 0.0.0.0 vlan1 direct 0/0

Related Commands

ip route

Notes

  • If no default route exists, then the message “Route not found” is printed

  • Route next hop is BFD controlled, status is viewable when <all> is inserted in the command, and it will be shown as follows:

    • If route is removed from routing decision it will be marked as “Active”

    • Protected next hops are marked with “B”

    • BFD protected failed/non active neighbors are marked with “BF”

ping

ping <interface> [-LRUbdfnqrvVaA] [-c count] [-i interval] [-w deadline] [-p pattern] [-s packetsize] [-t ttl] [-I interface or address] [-M mtu discovery hint] [-S sndbuf] [-T timestamp option ] [-Q tos ] [hop1 ...] destination

Sends ICMP echo requests to a specified host.

Syntax Description

Interface

Device/port (e.g., 1/2)

Linux Ping options

Default

N/A

Configuration Mode

config

History

18.01.5002

Example

metrox (config) # ping interface long-haul 1/2 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=0.456 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.222 ms
^C
--- 1.1.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1038ms
rtt min/avg/max/mdev = 0.222/0.339/0.456/0.117 ms

Related Commands

traceroute

Notes

When using -I option use the interface name + interface number, for example “ping -I vlan10”

traceroute

traceroute <interface> [-46dFITUnrAV] [-f first_ttl] [-g gate,...] [-i device] [-m max_ttl] [-N squeries] [-p port] [-t tos] [-l flow_label] [-w waittime] [-q nqueries] [-s src_addr] [-z sendwait] host [packetlen]

Traces the route packets take to a destination.

Syntax Description

Interface

Device/port (e.g., 1/2)

-4

Uses IPv4

-d

Enables socket level debugging

-F

Sets DF (“do not fragment” bit) on

-I

Uses ICMP ECHO for tracerouting

-T

Uses TCP SYN for tracerouting

-U

Uses UDP datagram (default) for tracerouting

-n

Does not resolve IP addresses to their domain names

-r

Bypasses the normal routing and send directly to a host on an attached network

-A

Performs AS path lookups in routing registries and print results directly after the corresponding addresses

-V

Prints version info and exit

-f

Starts from the first_ttl hop (instead from 1)

-g

Routes packets throw the specified gateway (maximum 8 for IPv4 and 127 for IPv6)

-i

Specifies a network interface to operate with

-m

Sets the max number of hops (max TTL to be reached)
Default: 30

-N

Sets the number of probes to be tried simultaneously
Default: 16

-p

Uses destination port. It is an initial value for the UDP destination port (incremented by each probe, default is 33434), for the ICMP seq number (incremented as well, default from 1), and the constant destination port for TCP tries (default is 80).

-t

Sets the TOS (IPv4 type of service) or TC (IPv6 traffic class) value for outgoing packets

-l

Uses specified flow_label for IPv6 packets

-w

Sets the number of seconds to wait for response to a probe (default is 5.0). Non-integer (float point) values allowed too.

-q

Sets the number of probes per each hop
Default: 3

-s

Uses source src_addr for outgoing packets

-z

Sets minimal time interval between probes (default is 0). If the value is more than 10, then it specifies a number in milliseconds, else it is a number of seconds (float point values allowed too).

Default

N/A

Configuration Mode

config

History

18.01.5002

Example

metrox (config) # traceroute interface long-haul 1/2 192.168.10.70
traceroute to 192.168.10.70 (192.168.10.70), 30 hops max, 40 byte packets
1 172.30.0.1 (172.30.0.1) 3.632 ms 2.849 ms 3.544 ms
2 10.222.128.46 (10.222.128.46) 3.176 ms 3.289 ms 3.656 ms
3 10.158.128.30 (10.158.128.30) 15.331 ms 15.819 ms 16.388 ms
4 10.158.128.65 (10.158.128.65) 20.468 ms 7.893 ms 12.27 ms
5 10.7.34.115 (10.7.34.115) 16.405 ms 11.985 ms 12.264 ms
6 192.168.10.70 (192.168.10.70) 16.377 ms 16.091 ms 20.475 ms

Related Commands

Notes

  • The following flags are not supported: -6, -l, -A

  • When using -i option use the interface name + interface number, for example “traceroute -i mgmt0”

tcpdump

tcpdump <interface> [-aAdeflLnNOpqRStuUvxX] [-c count] [-C file_size] [-E algo:secret] [-F file] [-i interface] [-M secret] [-r file] [-s snaplen] [-T type] [-w file] [-W filecount] [-y datalinktype] [-Z user] [expression]

Invokes standard binary, passing command line parameters straight through. Runs in foreground, printing packets as they arrive, until the user hits Ctrl+C.

Syntax Description

Interface

Device/port (e.g., 1/2)

Linux Ping options

Default

N/A

Configuration Mode

config

History

18.01.5002

Example

metrox (config) # tcpdump interface long-haul 1/2
......
09:37:38.678812 IP 192.168.10.7.ssh > 192.168.10.1.54155: P 1494624:1494800(176) ack 625 win 90
<nop,nop,timestamp 5842763 858672398>
09:37:38.678860 IP 192.168.10.7.ssh > 192.168.10.1.54155: P 1494800:1495104(304) ack 625 win 90
<nop,nop,timestamp 5842763 858672398>
...
9141 packets captured
9142 packets received by filter
0 packets dropped by kernel

Related Commands

Notes

  • When using -i option use the interface name + interface number, for example “tcpdump -i mgmt0”

  • For all flag options of this command refer to the linux ‘man page’ of tcp dump

show system state

show system state

Displays information on the state of the ConnectX cards (adapter cards) of the system.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

18.01.5002

Example

metrox (config) # show system state

System state : system-ready
System state description: system is ready to send/receive traffic

------------------------------------------------------------------------
Card ID Ports Card type Card state
------------------------------------------------------------------------
1 IB 1/1, LH 1/2 ConnectX7 card-ready
2 IB 2/1, LH 2/2 ConnectX7 card-ready
IB-MGMT IB-MGMT1 ConnectX7 card-ready

Related Commands

Notes

show asic-version

show asic-version

Displays FW versions of the ConnectX7 cards installed on the system

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

18.01.5002

Example

metrox (config) # show asic-version

-----------------------------------------------------
Device Type Version
-----------------------------------------------------
1 ConnectX7 28.0.99.1290
2 ConnectX7 28.0.99.1290
IB-MGMT ConnectX7 28.0.99.1290

Related Commands

Notes

© Copyright 2023, NVIDIA. Last updated on Sep 8, 2023.