P2P File

The P2P file is an Excel file that details the physical link connections within the fabric. It may consist of multiple sheets, each containing the following columns:

• A-Node Name: Specify the name of the node on the "A" side of the connection.

• A-Type: Indicate the role of the "A" side node, either "Host" or "Switch" (applicable for Ethernet).

• A-Port: Provide the port name for the "A" side node.

• Z-Node Name: Specify the name of the node on the "Z" side of the connection.

• Z-Type: Indicate the role of the "Z" side node, either "Host" or "Switch" (applicable for Ethernet).

• Z-Port: Provide the port name for the "Z" side node.

InfiniBand Example:

A sample sheet for InfiniBand connections in a P2P file:

• The designated port can be a single number or a split port (e.g., 1/2).

• Mapping for HCA ports:

  • 1 → mlx5_0 P1

  • 2 → mlx5_1 P1

  • And so on.

The HCA mapping could be customized by the user, for more details see HCA Mapping File

XDR Example:

A sample sheet for NVOS connections in a P2P file:

• Designated ports are represented as sw<port_number>p<split_number>for switches.

• Mapping for HCA ports: Same as InfiniBand (see above).

Ethernet Example:

Designated ports are represented as swp<port_number>s<split_number> for switches.

Please note that in all fabrics the tool relies on the header names to extract the information it needs. So user must have these names exactly as they appear above. If you make a syntax error then it will fail.

it is mandatory for the PTP file to incorporate a "Legend" sheet, which contains vital details regarding switch and host patterns. The below is an example:

Example:

CVT 1.5 introduced a n ew Unified Topology file format. This format is further enhanced in CVT 1.6. This format aims to manage various cluster types, including InfiniBand( HDR/NDR/XDR) , Ethernet, and NVLink, with ease using an Excel-based structure for organized data representation. The proposed format (Excel file) with multiple sheets encapsulates the topology of modern data center requirements and acts as a one-stop solution.

The Unified Topology file format consists of 4 sheets:

1. Nodes

2. Links

3. DC Floor Layout (optional sheet if not managing GB200/300 racks)

4. Server Profile (optional sheet if not managing Servers/Hosts)

Nodes

The Nodes sheet will focus on providing comprehensive information about the nodes within the cluster. The primary data points will include:

Notes:

• If Rack/Unit information is missing, some features—specifically the Rack View and filtering capabilities—will not function in the CVT interface.

• IB hosts need to be added to the nodes sheet, so that their corresponding links to switches can be detected. However, agent installation on them is not supported, hence their 'Managed' column value should be 'no'.

• Column names are case-insensitive, and the order of columns does not affect functionality.

• The user can dismiss optional columns as described in the Mandatory column.

Links

The Links sheet will detail the connectivity information across the nodes within the cluster. It will consist of the following columns:

In case of Host/Server ports, the A/Z-port would be the custom NIC name if available or the default NIC name. This value can be found in the output of `ip address show` command.

For IB hosts it is mandatory to have a server profile for this interface so that there is a mapping available for corresponding RDMA name of the port. RDMA name can be found from output of command: `sudo mst status -v`.

Notes:

• Column names are case-insensitive, and the order of columns does not affect functionality.

• The user can dismiss Optional Columns as explained in the Mandatory column.

Internal Links:

Internal links (NVLink within the GB200/300 racks) will not be part of the Links sheet. CVT shall detect any internal links based on the RackType and shall use the predefined JSON representation of these links to build the links. This approach ensures seamless integration and efficient configuration within the GB200/300 racks.

DC Floor Layout

The data center floor layout sheet is an optional sheet that is useful to provide a layout of the data center. This includes information about:

• Data Halls: The various halls within the data center.

• Scalable Units: Units designed to be scalable for future expansions. Add a default SU name for all if you are not using scalable unit concepts.

• Racks: Detailed information about the racks within the data center. This information is mandatory for GB200/300 racks.

• • Rack: Rack Name

• • Rack Type: Type of the rack. Can be GB200_72x1, GB200_36x2, or GB200_36x1 for GB200 racks (similar convention is used for GB300) or any other general rack type like ServerRack, NetworkRack for other general racks.

• • Rack Group: Used for grouping the racks, especially in GB200 racks. It is a unique number for each GB200 rack system. A GB200 72x1 system would have a unique group number for itself. In a GB200 36x2 system, the two racks will belong to the same group_number. (Group number is just a made-up integer to facilitate the identification of rack systems correctly).

Server Profile

The Server Profile sheet will be used to specify the interface configurations on Hosts, it does not have any significance for switches. The ServerProfile name mentioned in the Nodes sheet should have a corresponding entry here. Server Profile sheet will include below information: