Hardware Overview
The system supports the following:
Qty. | Interface |
x1 | On/Off button with an integral LED |
x2 | RJ-45 Ports |
x2 | 10Gb Ethernet Connectors |
x2 | XDR Ports (allow connectivity to an IB Switch FNM port) |
x2 | USB Ports |
Status LEDs | |
x2 | Hot-swap SSD Drawers |
Front Side Interfaces
Contains the system health, system ID, and status LED
Interface | Description | |
1 | Contains the system health, system ID, and status LED | |
2 | ConnectX-8 XDR single-port adapter card, allow connectivity to a switch FNM Port | |
3 | ConnectX-8 dual-port adapter card, allow connectivity to SN5600 INBAND | |
4 | NIC ports | 2x 10GbE Base-T ports |
5 | This port is designed for secure local and remote server management control (BMC) and helps IT administrators deploy, update and monitor the device. | |
6 | 2x USB 3.0-compliant | |
7 | Mini Display Port | Mini display port video connector |
8 | Drive slots | 2x E1.S NVMe hot-swap drives. Enables you to install drives that are supported on your system |
9 | Pull-Tab | A slide-out label panel containing system information (e.g., MAC address, serial number, etc.) |
Rear Side Interfaces
Index | Interface | Description |
1 | Bus-bar | Bus-bar Power Connector |
Control Panel View
Index | Indicator or Button | Icon | Description |
1 | Power Button | 
| The main power switch applies or removes primary power from the power supply to the server but maintains standby power. |
2 | UID Switch/LED/ BMC button | 
| The unit identification (UID) button turns on or off the blue light function of the Information LED and a blue LED on the rear of the chassis.This button can also be used to reset the BMC. |
3 | Power LED | 
| Steady on – Power on Blinking at 4Hz – Checking BIOS/BMC integrityBlinking at 4Hz and "i" LED is blue – BIOS firmware updatingTwo blinks at 4Hz, one pause 2hz and "i" LED blue – BMC firmware updatingBlinking at 1Hz and "i" LED red – Fault detected |
4 | SSD LED | 
| Indicates activity on the storage drives when flashing. |
5 | NIC LEDs | 
| Indicates network activity on the LAN ports when flashing |
6 | Information LED | 
| Alerts operator to several states (noted in the table below) |
7 | Power Fail LED | 
| Indicates a power supply module has failed. |
Information LEDs Specifications
Information LED | ||
Color | Status | Description |
Red | Solid | An overheating condition has occurred |
Red | Blinking at 1 Hz | Fan failure, check for an inoperative fan |
Red | Blinking at 10 Hz | CPLD recovery mode error |
Blue | Solid | Unit ID has been activated by switch |
Blue | Blinking at 1 Hz | Unit ID has been activated using the BMC |
Blue | Blinking at 2 Hz, and BMC Heartbeat LED on the motherboard is green | BMC is resetting |
Blue | Blinking at 4 Hz | BMC is setting factory defaults |
Blue | Blinking at 10 Hz | BIOS/BMC is recovering or updating |
Red | Blinking at 10 Hz and the rear UID LED is blue, blinking at 10 Hz | CPLD recovery or update is in progress |
Front Control Panel LED Specifications
Front Control Panel LED Indicators | ||||||
Event | Power | SSD | NIC | UID | Info | Power Fail |
Power On | Solid On | - | - | - | - | - |
SDD Activity | - | Blinking | - | - | - | |
NIC Activity | - | - | Blinking | - | - | - |
Overheat | - | - | - | - | Solid On | - |
Fan Fail | - | - | - | - | Blinking 1 Hz | - |
Power Fail | - | - | - | - | Blinking 1/4 Hz | Solid On |
Local UID On | - | - | - | Solid On | - | - |
Remote UID On | - | - | - | Blinking 1 Hz | - | - |
Checking BMC/BIOS | Blinking at 4 Hz | - | - | - | - | - |
Recovering/Updating |
| - | - | BIOS/BMC Blinking at 10 Hz | - | - |
Flash Not Detected or Golden Image Check Failed | BMC/BIOS Blinking at 1 Hz | - | - | - | - | - |
CPLD Recovery Mode | - | - | - | Blinking at 10 Hz (MB UID LED) | Blinking at 10 Hz (FP Red LED) | - |
Input/Output Panel
Index | Indicator or Button | Icon | Description |
1 | UID Switch/LED/BMC button | The unit identification (UID) button turns on or off the blue light function of the Information LED and a blue LED on the rear of the chassis. This button can also be used to reset the BMC. | |
2 | LAN Port | RJ45 port is shared for system network access and BMC access; for indicator details, see BMC LAN LEDs | |
3 | USB Ports | 2x USB 3.0 | |
4 | Mini Display Port | Mini Display Port video connector |
SSD LED Indicators
Each E1.S SSD has two LED indicators: an activity indicator and a status indicator.
Drive Carrier LED Specifications
LED | Color | Blinking Pattern | Behavior for Device |
Activity LED | Green | Solid On | Idle |
Green | Blinking | I/O activity | |
Off | - | Eject device | |
Status LED | Off | - | Idle |
Amber | Blinking at 4 Hz | Locate device | |
Amber | Blinking at 1 Hz for a few seconds, then off | Eject device or insert device, as appropriate |
2x ConnectX-8 OSFP SuperNIC
These two OSFP ports are found on the front side of the appliance (see callout 4 and 9 in "Front Side Interfaces" figure above). and allow connectivity to an FNM ( Fabric Network Management) InfiniBand switch port in the fabric. It is recommended to connect to two different switches for redundancy. The appliance can be connected only to a single IB fabric.
Connecting the UFM XDR Appliance to the FNM InfiniBand Switch Port
The FNM port is a separate OSFP InfiniBand in-band management port. It enables accessing the UFM Enterprise appliance that allows data center operators to efficiently monitor and operate the entire fabric .
Since the FNM port is semi-populated (only the first four lanes of it are wired), it functions as an HDR port.
To support this connectivity, change the default configuration of the ConnectX-8 XDR adapter card by running:
mlxconfig -d <device> set NUM_OF_PLANES_P1=1sudo reboot
By default, the ConnectX®-8 XDR adapter is configured for 4 planes, as it is typically connected to an XDR multi-plane switch. Since the NDR switch is single-plane, this parameter must be set to 1 to ensure proper operation with the NDR switch.
XDR Cluster
Switch | Cable / Transceiver SKU on Switch Side | Cable / Transceiver SKU on UFM XDR-HCA Side* |
Quantum-3 Q3200 Switch Network Port | MMS4X00-NS / 980-9I30H-00NM00 (100m) | MMS4X00-NS400 / 980-9I31N-00NM00 (100m) |
Quantum-X800 Q3400 Switch Network Port | MMS4X00-NS / 980-9I30H-00NM00 (100m) | MMS4X00-NS400 / 980-9I31N-00NM00 (100m) |
Quantum-X800 Q3400 Switch FNM as XDR Lite (as 4 1x 100G) | MMS4X00-NS / 980-9I30H-00NM00 (100m) | MMS4X00-NS400 / 980-9I31N-00NM00 (100m) |
* UFM XDR Appliance side should configure the ConnectX-8 C8180 cards to be non-planarized. ** The 980-9I30H-00NM00 transceiver supports a 2x100 Gb/s configuration . | ||
NDR Cluster
To support this connectivity, change the default configuration of all the ConnectX®-8 XDR adapter cards by running the following commands via the SHELL:
mlxconfig -d <device> set NUM_OF_PLANES_P1=1
sudo reboot
Alternatively, run the following CLI command:
ib profile --set ndr --force
By default, the ConnectX®-8 XDR adapters are configured for 4 planes, as they are typically connected to an XDR multi-plane switch. Since the NDR switch is single-plane, this parameter must be set to 1 to ensure proper operation with the NDR switch.
Switch | Cable / Transceiver SKU on Switch Side | Cable / Transceiver SKU on UFM XDR-HCA Side* |
Quantum-2 MQM97xx Network Port | MMA4Z00-NS / 980-9I510-00NS00 (50m) | MMA4Z00-NS400 / 980-9I51S-00NS00 (50m) |
Quantum-2 MQM97xx Network Port | MMS4X00-NS / 980-9I30H-00NM00 (100m) | MMS4X00-NS400 / 980-9I31N-00NM00 (100m) |
* UFM XDR Appliance side should configure the ConnectX-8 C8180 cards to be non-planarized. ** The 980-9I30H-00NM00 / 980-9I510-00NS00 transceivers support a 2x100 Gb/s configuration . | ||
1x ConnectX-8 QSFP SuperNIC
There is one dual QSFP ConnectX-8 SuperNIC located on the front side of the appliance (see callout 10 in "Front Side Interfaces" figure above) which allows Ethernet connectivity via the Inband connection to two NVIDIA Spectrum-4 SN5600 Switch Systems.
NIC Ports
Activity LED Indicators
Each NIC on the back of the system has indicators that provide information about the activity and link status. The activity LED indicator indicates if data is flowing through the NIC, and the link LED indicator indicates the speed of the connected network.
NIC Activity LEDs
Index | Description |
1 | Link LED indicator |
2 | Activity LED indicator |
The following table lists the drive indicator codes:
NIC Indicator Code | Condition |
Link and activity indicators are off | Indicates that the NIC is not connected to the network |
Link indicator is green, and activity indicator is blinking green | Indicates that the NIC is connected to a valid network at its maximum port speed, and data is being sent or received |
Link indicator is amber, and activity indicator is blinking green | Indicates that the NIC is connected to a valid network at less than its maximum port speed, and data is being sent or received |
Link indicator is green, and activity indicator is off | Indicates that the NIC is connected to a valid network at its maximum port speed, and data is not being sent or received |
Link indicator is amber, and activity indicator is off | Indicates that the NIC is connected to a valid network at less than its maximum port speed, and data is not being sent or received |
Link indicator is blinking green, and activity is off | Indicates that the NIC identity is enabled through the NIC configuration utility |
RJ-45 Port
The remote management (BMC) control port is designed for secure local and remote server management and helps IT administrators deploy, update, and monitor the NVIDIA® UFM® Enterprise Appliance.
RJ-45 Management Ports
These four RJ-45 ports (eno8303, eno8403) are located on the front side of the appliance (see callout 5 in "Front Side Interfaces" figure above). The eno8303 and the remote management interfaces are pre-configured as DHCP and the initial host name is ubuntu, so their IP addresses can be obtained from the DHCP server. If no DHCP server is available, you have to use a serial cable to connect and configure eno8303 and the remote-management IP addresses with a static IP address.
Configuring the appliance via the serial port is required only in the case where out-of-the-box DHCP configuration for eno8303 cannot be used. (There is no DHCP server in the management network). The user is then required to use the serial port to configure a static IP on eno8303.
NIC#1 Ethernet connector gets connected to Ethernet switches. This switch must be configured to 100M/1G auto-negotiation.
RJ-45 Ethernet Connector for Remote Management (BMC)
The appliance has several Ethernet management interfaces. The primary management interface is eno8303. An additional interface exists, for connecting to a remote management controller (BMC) (It usually connects to the same management network as eno8303).
To use out-of-the-box DHCP settings, the default hostname for the appliance (over eno8303) is "ubuntu". The MAC address for eno8303 is available on the pull-tab and can be configured in the DHCP server.
To use the remote management controller (BMC) with DHCP, the free-range IP allocation must be enabled on the DHCP server. A static IP address for remote management interface (BMC) can be configured via the Remote Management (BMC) web application.
For more information about RJ-45 Ethernet Remote Management, (BMC) please refer to Appendix – Remote Access Controller.
Configuration via a serial port is only required if you want to use a static IP address and not the out-of-the-box DHCP setting for eno8303. Otherwise, an IP is assigned by the DHCP server, and you can log into the CLI over LAN.
NIC#1 Ethernet connector gets connected to Ethernet switches. This switch must be configured to 100M/1G auto-negotiation.
USB Interface
The USB interface can be used to update the UFM.
There are two USB connectors. These connectors can be used to install software and/or firmware upgrades using a memory device that has a USB connector. This connector is USB 2.0 compliant. Various upload/download operations are also supported through the USB using the CLI such as:
UFM configuration fetching/uploading
UFM license fetching
UFM upgrade
All USB connectors can be used to perform SW updates or various upload/download operations using the CLI.
Input/Output Panel
Index | Indicator or Button | Icon | Description |
1 | UID Switch/LED/BMC button | The unit identification (UID) button turns on or off the blue light function of the Information LED and a blue LED on the rear of the chassis. This button can also be used to reset the BMC. | |
2 | LAN Port | RJ45 port is shared for system network access and BMC access; for indicator details, see BMC LAN LEDs | |
3 | USB Ports | 2x USB 3.0 | |
4 | Mini Display Port | Mini Display Port video connector |