Configuring the Appliance for the First Time
The diagram below describes the connectivity scheme of the UFM High-Availability cluster.
The following are instructions on how to configure the management and fabric (InfiniBand) interfaces in the UFM cluster.
The NVIDIA UFM Enterprise Appliance has multiple Ethernet management interfaces. The primary management interface isenp99s0f0. The MAC address for enp99s0f0 is available on the pull tab and can be configured in the DHCP server. To use the remote management controller with DHCP, the free-range IP allocation must be enabled on the DHCP server.
The appliance supports a direct connection via a serial port.
For instructions on how to configure the management interface, please refer to Configuring the Appliance.
This interface should be used as the primary interface when configuring HA.
When operating in HA configuration, directly connect (back-to-back - without a management switch in the middle) the Master node to the Standby node. To do so, utilize the Ethernet management interface enp99s0f1, as shown in the above diagram.
For your convenience, you may use the CLI command Interface to set a static IP address for enp99s0f1.
Example:
interface
enp99s0f1 ip address 11.0
.0.11
/24
Configuring the fabric interface is optional.
The UFM XDR Enterprise Appliance has multiple InfiniBand interfaces. The primary interface is ib0.
Configure a static IPoIB with Network service (create the file /etc/network/interfaces.d/ifcfg-ib0
and run ifup ib0
).
Example of the ifcfg-ib0
file definition:
auto ib0
iface ib0 inet static
address 10.0
.0.12
netmask 255.255
.255.0
broadcast 10.0
.0.255
For your convenience, you may use the CLI command Interface to set a static IP address for ib0.
Example:
interface
ib0 ip address 192.168
.1.11
/24
For more details on how to configure the UFM Enterprise, please refer to UFM Enterprise Initial Configuration.
The FNM (Fabric Network Management) port is a separate OSFP InfiniBand in-band management port. It enables accessing the UFM Enterprise appliance that allows data center operators to efficiently monitor and operate the entire fabric.
Since the FNM port is semi-populated (only the first four lanes of it are wired), it functions as an HDR port.
To support this connectivity, change the default configuration of the ConnectX®-8 XDR adapter card by running:
mlxconfig -d <device> set NUM_OF_PLANES_P1=1
sudo reboot
By default, the ConnectX®-8 XDR adapter is configured for 4 planes, as it is typically connected to an XDR multi-plane switch. Since the NDR switch is single-plane, this parameter must be set to 1 to ensure proper operation with the NDR switch.
Q3200 Air Cooled - Non-planarized FNM Port (Hardware Limited to 2x100 Configuration) | ||||
Mode | Description | Transceivers Legacy Part Number | Transceivers SKU | |
NDR SM - Single Mode** | FNM side | 1 port is active in 200Gb/s, switch side (Finn) | 980-9I30H-00NM00 | 980-9I30H-00NM00 (100m) |
NDR SM - Single Mode** | UFM XDR Appliance side | 1 port in 200Gb/s (RHS) | MMS4X00-NS400 | 980-9I31N-00NM00 (100m) |
NDR MM** | FNM side | 1 port is active in 200Gb/s, switch side (Finn) | MMA4Z00-NS | 980-9I510-00NS00 (50m) |
NDR MM* | UFM XDR Appliance side | 1 port in 200Gb/s (RHS) | MMA4Z00-NS400 | 980-9I51S-00NS00 (50m) |
XDR SM -Single Mode** | FNM side | 1 port is active in 200Gb/s, switch side (IHS) | - | 980-9IAH1-00XM00 (500m) |
XDR SM - Single Mode* | UFM XDR Appliance side | 1 port in 200Gb/s (RHS) | - | 980-9IAI0-00XM00 (500m) |
|
Q3400 Air Cooled - Planarized FNM Port | ||||
Mode | Description | Transceivers Legacy Part Number | Transceivers SKU | |
NDR - Single Mode | FNM side | 1 port is active in 400Gb/s, switch side (Finn) | MMS4X00-NS | 980-9I30H-00NM00 (100m) |
NDR - Single Mode | UFM XDR Appliance side | 1 port in 400Gb/s (RHS) | MMS4X00-NS400 | 980-9I31N-00NM00 (100m) |
NDR Multi Mode | FNM side | 1 port is active in 400Gb/s, switch side (Finn) | MMA4Z00-NS | 980-9I510-00NS00 (50m) |
NDR Multi Mode | UFM XDR Appliance side | 1 port in 400Gb/s (RHS) | MMA4Z00-NS400 | 980-9I51S-00NS00 (50m) |
XDR - Single Mode* | FNM side | 1 port is active in 400Gb/s, switch side (IHS) | - | 980-9IAH1-00XM00 (500m) |
XDR - Single Mode* | UFM XDR Appliance side | 1 port in 400Gb/s (RHS) | - | 980-9IAI0-00XM00 (500m) |
* The 980-9IAH1-00XM00 transceiver supports a 4x100 Gb/s configuration with a planarized port design. |