ibdiagnet InfiniBand Fabric Diagnostic Tool User Manual v2.8.0

Dragonfly+ Topology Validation

This section specifies the options for DragonFly+ topology validation. Topology validation checks that all DFP+ islands are built as two level “fat-trees” and they are properly interconnected. It also reports on a network theoretical “bisectional” bandwidth of the topology.

A newly generated output file – ibdiagnet2.dfp will contain details about DFP islands, their root switches, connectivity tables, global links and theoretical bandwidth.

Parameter

Description

--dfp

Provides a report of the fabric Dragonfly+ analysis.

Data will be dumped to the ibdiagnet2.dfp file.

Warnings and errors will be dumped to the ibdiagnet2.log file.

--dfp_opt

Comma separated Dragonfly+ validation options.

Possible values are:

  • max_cas=<num> - maximum number of CA on Dragonfly+ islands root switches (default=1). This parameter is mutual exclusive with –smdb parameter

--smdb <file>

Loads DFP+ islands roots from the subnet manager SMDB file (if routing engine reported in opensm-smdb.dump file is “dfp”).

Example:

Copy
Copied!
            

ibdiagnet –-dfp

  • ibdiagnet's Output:

    Copy
    Copied!
                

    DFP Topology Validation -I- 5 DFP islands were detected -I- DFP Symmetrical switch connectivity discovered, global links per island: 16 -I- Partially resilient DFP discovered -I- All DFP islands have the same bandwidth: 160 Gbps -I- Theoretical DFP network bisection bandwidth: 107 Gbps -W- DFP large topology was detected; may be implemented as medium -W- DFP validation finished with warnings

  • dfp file islands details:

    Copy
    Copied!
                

    island: 0  rank: 0 (size: 4)         0x0002c90000000194 R0_G0_S1/U1         0x0002c90000000198 R0_G0_S3/U1         0x0002c900000001cc R1_G0_S0/U1         0x0002c900000001d0 R1_G0_S2/U1    rank: 1 (size: 4)         0x0002c90000000004 G0_L0/U1         0x0002c90000000018 G0_L1/U1         0x0002c9000000002c G0_L2/U1         0x0002c90000000040 G0_L3/U1

  • dfp file islands connectivity table:

    Copy
    Copied!
                

    island 0, bandwidth 160 [Gb/s], global links 16, resilient connection to all: No         switch 0x0002c90000000194, global links: 4, resilient connection to all: No, connected islands: 2, free ports: 73                 island: 2, global links: 3, resilient: Yes                 island: 3, global links: 1, resilient: No         switch 0x0002c90000000198, global links: 4, resilient connection to all: No, connected islands: 4, free ports: 73                 island: 2, global links: 1, resilient: No                 island: 1, global links: 1, resilient: No                 island: 3, global links: 1, resilient: No                 island: 4, global links: 1, resilient: No         switch 0x0002c900000001d0, global links: 4, resilient connection to all: No, connected islands: 3, free ports: 73                 island: 1, global links: 1, resilient: No                 island: 3, global links: 2, resilient: No                 island: 4, global links: 1, resilient: No         switch 0x0002c900000001cc, global links: 4, resilient connection to all: No, connected islands: 2, free ports: 73                 island: 1, global links: 2, resilient: No                 island: 4, global links: 2, resilient: No

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.