ConnectX-3 onwards adapter cards’ ports can be individually configured to work as InfiniBand or Ethernet ports. By default, ConnectX family adapter cards VPI ports are initialized as InfiniBand ports. If you wish to change the port type use the mstconfig after the driver is loaded.
Install mstflint
zypper install mstflint
Check the PCI
spci | grep Mellanox
Example:
24
:00.0
Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5
Ex]Use mstconfig to change the link type as desired IB (for InfiniBand) or ETH (for Ethernet).
mstconfig –d <device pci> s LINK_TYPE_P1/
2
=<ETH|IB|VPI>Example:
# mstconfig -d
00
:06.0
s LINK_TYPE_P1=ETH Device #1
--------- Device type: ConnectX5 Name: MCX556A-EDA_Ax Description: ConnectX-5
Ex VPI adapter card; EDR IB (100Gb/s) and 100GbE; dual-port QSFP28; PCIe4.0
x16; tall bracket; ROHS R6 Device:24
:00.0
Configurations: Next Boot New LINK_TYPE_P1 IB(1
) ETH(2
) Applynew
Configuration? (y/n) [n] : y Applying... Done! -I- Please reboot machine to loadnew
configurations.Reboot the machine.
Query the device’s parameters to validate the new configuration.
# mstconfig -d
00
:06.0
q Device #1
: ---------- Device type: ConnectX5 Name: MCX556A-EDA_Ax Description: ConnectX-5
Ex VPI adapter card; EDR IB (100Gb/s) and 100GbE; dual-port QSFP28; PCIe4.0
x16; tall bracket; ROHS R6 Device:24
:00.0
Configurations: Next Boot MEMIC_BAR_SIZE0
MEMIC_SIZE_LIMIT _256KB(1
) HOST_CHAINING_MODE DISABLED(0
) HOST_CHAINING_DESCRIPTORS Array[0
..7
] HOST_CHAINING_TOTAL_BUFFER_SIZE Array[0
..7
] FLEX_PARSER_PROFILE_ENABLE0
FLEX_IPV4_OVER_VXLAN_PORT0
ROCE_NEXT_PROTOCOL254
ESWITCH_HAIRPIN_DESCRIPTORS Array[0
..7
] ESWITCH_HAIRPIN_TOT_BUFFER_SIZE Array[0
..7
] NON_PREFETCHABLE_PF_BAR False(0
) NUM_OF_VFS4
SRIOV_EN True(1
) PF_LOG_BAR_SIZE5
VF_LOG_BAR_SIZE1
NUM_PF_MSIX63
NUM_VF_MSIX11
INT_LOG_MAX_PAYLOAD_SIZE AUTOMATIC(0
) SW_RECOVERY_ON_ERRORS False(0
) RESET_WITH_HOST_ON_ERRORS False(0
) ADVANCED_POWER_SETTINGS False(0
) CQE_COMPRESSION BALANCED(0
) IP_OVER_VXLAN_EN False(0
) PCI_ATOMIC_MODE PCI_ATOMIC_DISABLED_EXT_ATOMIC_ ENABLED(0
) LRO_LOG_TIMEOUT06
LRO_LOG_TIMEOUT17
LRO_LOG_TIMEOUT28
LRO_LOG_TIMEOUT313
LOG_DCR_HASH_TABLE_SIZE11
DCR_LIFO_SIZE16384
LINK_TYPE_P1 ETH(2
) LINK_TYPE_P2 IB(1
) ROCE_CC_PRIO_MASK_P1255
ROCE_CC_ALGORITHM_P1 ECN(0
) ROCE_CC_PRIO_MASK_P2255
ROCE_CC_ALGORITHM_P2 ECN(0
) CLAMP_TGT_RATE_AFTER_TIME_INC_P1 True(1
) CLAMP_TGT_RATE_P1 False(0
) RPG_TIME_RESET_P1300
RPG_BYTE_RESET_P132767
RPG_THRESHOLD_P11
RPG_MAX_RATE_P10
RPG_AI_RATE_P15
RPG_HAI_RATE_P150
RPG_GD_P111
RPG_MIN_DEC_FAC_P150
RPG_MIN_RATE_P11
RATE_TO_SET_ON_FIRST_CNP_P10
DCE_TCP_G_P11019
DCE_TCP_RTT_P11
RATE_REDUCE_MONITOR_PERIOD_P14
INITIAL_ALPHA_VALUE_P11023
MIN_TIME_BETWEEN_CNPS_P12
CNP_802P_PRIO_P16
CNP_DSCP_P148
CLAMP_TGT_RATE_AFTER_TIME_INC_P2 True(1
) CLAMP_TGT_RATE_P2 False(0
) RPG_TIME_RESET_P2300
RPG_BYTE_RESET_P232767
RPG_THRESHOLD_P21
RPG_MAX_RATE_P20
RPG_AI_RATE_P25
RPG_HAI_RATE_P250
RPG_GD_P211
RPG_MIN_DEC_FAC_P250
RPG_MIN_RATE_P21
RATE_TO_SET_ON_FIRST_CNP_P20
DCE_TCP_G_P21019
DCE_TCP_RTT_P21
RATE_REDUCE_MONITOR_PERIOD_P24
INITIAL_ALPHA_VALUE_P21023
MIN_TIME_BETWEEN_CNPS_P22
CNP_802P_PRIO_P26
CNP_DSCP_P248
LLDP_NB_DCBX_P1 False(0
) LLDP_NB_RX_MODE_P1 OFF(0
) LLDP_NB_TX_MODE_P1 OFF(0
) LLDP_NB_DCBX_P2 False(0
) LLDP_NB_RX_MODE_P2 OFF(0
) LLDP_NB_TX_MODE_P2 OFF(0
) DCBX_IEEE_P1 True(1
) DCBX_CEE_P1 True(1
) DCBX_WILLING_P1 True(1
) DCBX_IEEE_P2 True(1
) DCBX_CEE_P2 True(1
) DCBX_WILLING_P2 True(1
) KEEP_ETH_LINK_UP_P1 True(1
) KEEP_IB_LINK_UP_P1 False(0
) KEEP_LINK_UP_ON_BOOT_P1 False(0
) KEEP_LINK_UP_ON_STANDBY_P1 False(0
) KEEP_ETH_LINK_UP_P2 True(1
) KEEP_IB_LINK_UP_P2 False(0
) KEEP_LINK_UP_ON_BOOT_P2 False(0
) KEEP_LINK_UP_ON_STANDBY_P2 False(0
) NUM_OF_VL_P1 4_VLs(3
) NUM_OF_TC_P1 _8_TCs(0
) NUM_OF_PFC_P18
NUM_OF_VL_P2 _4_VLs(3
) NUM_OF_TC_P2 _8_TCs(0
) NUM_OF_PFC_P28
DUP_MAC_ACTION_P1 LAST_CFG(0
) SRIOV_IB_ROUTING_MODE_P1 LID(1
) IB_ROUTING_MODE_P1 LID(1
) DUP_MAC_ACTION_P2 LAST_CFG(0
) SRIOV_IB_ROUTING_MODE_P2 LID(1
) IB_ROUTING_MODE_P2 LID(1
) PCI_WR_ORDERING per_mkey(0
) MULTI_PORT_VHCA_EN False(0
) PORT_OWNER True(1
) ALLOW_RD_COUNTERS True(1
) RENEG_ON_CHANGE True(1
) TRACER_ENABLE IPv4(0
) IP_VER0
BOOT_UNDI_NETWORK_WAIT False(0
) UEFI_HII_EN False(0
) BOOT_DBG_LOG DISABLED(0
) UEFI_LOGS1
BOOT_VLAN PXE(1
) LEGACY_BOOT_PROTOCOL NONE(0
) BOOT_RETRY_CNT1 True(1
) BOOT_LACP_DIS False(0
) BOOT_VLAN_EN0
BOOT_PKEY False(0
) EXP_ROM_PXE_ENABLE True(1
) IBM_TUNNELED_ATOMIC_EN False(0
) IBM_AS_NOTIFY_EN False(0
) ADVANCED_PCI_SETTINGS False(0
) SAFE_MODE_THRESHOLD10
SAFE_MODE_ENABLE True(1
) ******************************************************************