Routers

This section specifies options for retrieving InfiniBand Routers configuration and verification of Floating LIDs (FLIDs) configuration on a local and adjacent networks. If FLIDs are enabled, a newly generated ibdiagnet.flid file will contain details about networks FLIDs ranges, FLIDs enabled on the routers, and distribution of FLIDs per switches in the local network.

Parameter

Description

--routers

Discovers and dumps router tables to ibdiagnet2.db_csv file, and runs FLIDs verification.

Examples:

Copy
Copied!
            

ibdiagnet –routers

  • ibdiagnet's Output: FLIDs are disabled/subnet does not have any router

    Copy
    Copied!
                

    Routers -I- Build Routers Info DB finished successfully -I- Build Routers Tables finished successfully -I- Adjacent subnets FLID Table retrieving finished successfully -I- Routers FLID Table retrieving finished successfully -I- Skipping FLID verification

  • ibdiagnet's Output: FLIDS are enabled on some routers

    Copy
    Copied!
                

    Routers -I- Build Routers Info DB finished successfully -I- Build Routers Tables finished successfully -I- Adjacent subnets FLID Table retrieving finished successfully -I- Routers FLID Table retrieving finished successfully   -I- All routers in the subnet have the same global FLID range: start=10 end=1100 -I- All routers in the subnet have the same local FLID range: start=20 end=500 -I- Local FLID range is in the global one. -I- Local subnet LID and global FLID ranges are OK. -I- Local subnet FLID verification finished successfully   -I- Ranges in the subnet: 0x233c are OK -E- Adjacent subnets FLID verification finished with errors -E- Different FLID ranges were detected for the subnet: 0x4789 [0x0002c90000000053, 0x0002c90000000053, 0x0002c9000000004a](total 3), start=583 end=667   -I- HCA and Switches FLID verification finished successfully

  • flid file: global, local and adjacent subnets ranges

    Copy
    Copied!
                

    Global FLID range: start=10 end=1100 Local subnet FLID range: start=20 end=500 Local subnet LID and global FLID ranges are OK   ------------------------------------------------------ Adjacent subnets Subnet: 0x0001 FLID range: start=20 end=500 Subnet: 0x233c FLID range: start=501 end=582 Subnet: 0x4789 different FLID ranges found on routers: [0x0002c90000000053, 0x0002c90000000053, 0x0002c9000000004a](total 3), start=583 end=667 [0x0002c90000000041](total 1), start=583 end=700

  • flid file: local and non-local FLIDs enabled on a router

    Copy
    Copied!
                

    Routers 0x0002c90000000041 - "Router1/RT", enabled FLIDs   local: 20 22 28 30 31 42 …..   non-local: 10 13 16 17 19 501 503 505 …..

  • flid file: FLIDs per switches

    Copy
    Copied!
                

    FLID per switches 0x0002c90000000064 - "SW-1-0/U1": 20 0x0002c90000000066 - "SW-1-1/U1": 21 0x0002c90000000068 - "SW-1-2/U1": 22 0x0002c9000000006a - "SW-1-3/U1": 23

When MKEY protection is enabled by the Subnet Manager on the fabric devices, following command options should be used in ibdiagnet:

Parameter

Description

--m_key <m_key>

Specifies constant MKey for the fabric. The MKey value should be specified when a single MKEY is shared by all InfiniBand devices.

The mkey value can be obtained from the opensm.conf file (m_key parameter).

--m_key_files <path to key files directory>

Specifies the path to the directory with the key_files (guid2lid, guid2mkey, neighbors, guid2cckey, guid2vskey).

Key files usually provided by opensm in /var/cache/opensm/.

Example of guid2mkey/guid2cckey/guid2vskey file:

Copy
Copied!
            

0x0002c9000000001e 0x0000000000000111 0x0002c9000000002a 0x0000000000000222 0x0002c90000000026 0x0000000000000333 0x0002c90000000006 0x0000000000000444

Example of guid2lid file:

Copy
Copied!
            

0x0002c9000000004b 0x0027 0x0027 0x0002c9000000002a 0x001a 0x001b 0x0002c90000000006 0x0004 0x0005 0x0002c90000000047 0x000e 0x000e

Example of neighbours file:

Copy
Copied!
            

0x0002c9000000004d:4 0x0002c9000000000e:1 0x0002c9000000004b:1 0x0002c9000000002e:1 0x0002c90000000049:2 0x0002c90000000022:1 0x0002c90000000006:1 0x0002c9000000004d:2

Example:

Copy
Copied!
            

ibdiagnet --mkey 0x00ff ibdiagnet --m_key_files /var/cache/opensm/ ibdiagnet --m_key_files /tmp/opensm_mkey_files/

Aggregation Management Key (AM key) is sent in SHARP Management MADs to the Aggregation nodes. (default 0). Aggregation management key is configured in SHARP Aggregation Manager configuration file.

Parameter

Description

--am_key <am_key>

Specifies constant am_key.
AM Key parameter value format: 0x<hex_value>

--am_key_file <path_to_am_key_file>

Specifies the path to am_key_file: guid2am_key.
The guid2am_key file includes pair of values GUID and key in the following format:
0x<guid_hex_value> 0x<am_key_hex_value>

Example of guid2amkey file:

Copy
Copied!
            

0x0000000000000111 0x0000000000000001 0x0000000000000222 0x2 0x0000000000000333 0x0000000003 0x0000000000000444 0x0000000000000004

Example:

Copy
Copied!
            

ibdiagnet --am_key 0x0000000000000123 ibdiagnet --am_key 0x123 ibdiagnet --am_key_file /tmp/guid2am_key ibdiagnet --am_key_file /tmp/am_keys

When VendorSpecific keys are generated and provisioned by the Subnet Manager, VendorSpecific keys should be sent in VendorSpecific MADs per endport.

Parameter

Description

--vs_key <vs_key>

Specifies constant vs_key.
VendorSpecific Key parameter value format: 0x<hex_value>

--vs_key_file <path_to_vs_key_file>

Specifies the path to vs_key_file: guid2vs_key.
The guid2vs_key file includes pair of values GUID and key in the following format:
0x<guid_hex_value> 0x<vs_key_hex_value>

Example of guid2vs_key FILE:

Copy
Copied!
            

0x0000000000000111 0x0000000000000001 0x0000000000000222 0x2 0x0000000000000333 0x0000000003 0x0000000000000444 0x0000000000000004

Example:

Copy
Copied!
            

ibdiagnet --vs_key 0x0000000000000123 ibdiagnet --vs_key 0x123 ibdiagnet --vs_key_file /tmp/guid2vs_key ibdiagnet --vs_key_file /tmp/vs_keys

When CongestionControl keys are generated and provisioned by the Subnet Manager, CongestionControl keys should be sent in CongestionControl (CC) MADs perend port.

Parameter

Description

--cc_key <cc_key>

Specifies constant cc_key.
CC Key parameter value format: 0x<hex_value>

--cc_key_file <path_to_cc_key_file>

Specifies the path to cc_key_file: guid2cc_key.
The guid2cc_key file includes pair of values GUID and key in the following format:
0x<guid_hex_value> 0x<cc_key_hex_value>

Example of guid2cc_key FILE:

Copy
Copied!
            

0x0000000000000111 0x0000000000000001 0x0000000000000222 0x2 0x0000000000000333 0x0000000003 0x0000000000000444 0x0000000000000004

Example:

Copy
Copied!
            

ibdiagnet --cc_key 0x0000000000000123 ibdiagnet --cc_key 0x123 ibdiagnet --cc_key_file /tmp/guid2cc_key ibdiagnet --cc_key_file /tmp/cc_keys

When Manager2Node keys are generated and provisioned by the Subnet Manager, Manager2Node keys should be sent in M2N (pFRN) MADs per node.

Parameter

Description

--m2n_key <m2n_key>

Specifies constant m2n_key for the fabric.
M2N Key parameter value format: 0x<hex_value>

--m2n_key_file <path_to_m2n_key_file>

Specifies the path to m2n_key_file: guid2_m2n_key.

The guid2_m2n_key file includes pair of values GUID and key in the following format:

0x<guid_hex_value> 0x<m2n_key_hex_value>

Example of guid2_m2n_key FILE:

Copy
Copied!
            

0x0000000000000111 0x0000000000000001 0x0000000000000222 0x2 0x0000000000000333 0x0000000003 0x0000000000000444 0x0000000000000004

Examples:

Copy
Copied!
            

ibdiagnet --m2n_key 0x0000000000000123 ibdiagnet --m2n_key 0x123 ibdiagnet --m2n_key_file /tmp/guid2_m2n_key ibdiagnet --m2n_key_file /tmp/m2n_keys

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.