NVIDIA WinOF-2 Documentation v3.10.52010 LTS
Linux Kernel Upstream Release Notes v6.5

General Capabilities

General supported capabilities:

Warning

The capabilities described below are applicable to both Ethernet and InfiniBand networks.

For retrieving the port types, perform one of the following:

  • Run mlx5cmd -stat from the "Command Prompt", and check the link_layer from the output.

  • See the Port Type in the Information tab in the Device Manager window (see Displaying Adapter Related Information)

To configure the port types to Ethernet/InfiniBand mode on a device, use the mlxconfig.exe utility, which is a part of the MFT package for Windows, and is available at https://network.nvidia.com/products/adapter-software/firmware-tools/.

  1. Install the WinMFT package.

  2. Retrieve the device name:

    1. In command prompt, run "mst status -v":

      Copy
      Copied!
                  

      mst status -v MST devices: ------------ mt4099_pci_cr0 bus:dev.fn=04:00.0 mt4099_pciconf0 bus:dev.fn=04:00.0 mt4103_pci_cr0 bus:dev.fn=21:00.0 mt4103_pciconf0 bus:dev.fn=21:00.0   mt4115_pciconf0 bus:dev.fn=24:00.0

    2. Identify the desired device by its "bus:dev.fn" PCIe address.

  3. Configure the port type to either InfiniBand or Ethernet:

    1. Ethernet, execute the following command with the appropriate device name:

      Copy
      Copied!
                  

      mlxconfig -d mt4115_pciconf0 set LINK_TYPE_P1=2

    2. InfiniBand, execute the following command with the appropriate device name:

      Copy
      Copied!
                  

      mlxconfig -d mt4115_pciconf0 set LINK_TYPE_P1=1

      Warning

      To set the type of the second port, set the parameter LINK_TYPE_P2.

  4. Reboot the system.

    Warning

    Changing the port type will change some of the registry keys to the default values of the new port type.

For further information, please refer to the MFT User Manual.

By default, your machine is configured to obtain an automatic IP address via a DHCP server. In some cases, the DHCP server may require the MAC address of the network adapter installed in your machine.

  1. Open a CMD console-> Click Start-> Task Manager-> File-> Run new task-> and enter CMD.

  2. Display the MAC address as “Physical Address”.

    Copy
    Copied!
                

    ipconfig /all

  1. Open the Network Connections window. Locate Local Area Connections with NVIDIA® devices.

    local_area_connections_page.PNG

  2. Right-click a NVIDIA® Local Area Connection and left-click Properties.

    local_area_connection_properties.png

  3. Select Internet Protocol Version 4 (TCP/IPv4) from the scroll list and click Properties.

  4. Select the “Use the following IP address:” radio button and enter the desired IP information.

    IP_address_setting.PNG

  5. Click OK.

  6. Close the Local Area Connection dialog.

  7. Verify the IP configuration by running ‘ipconfig’ from a CMD console.

    Copy
    Copied!
                

    ipconfig ... Ethernet adapter Local Area Connection 4:   Connection-specific DNS Suffix . : IP Address. . . . . . . . . . . . : 11.4.12.63 Subnet Mask . . . . . . . . . . . : 255.255.0.0 Default Gateway . . . . . . . . . : ...

  1. Open Device Manager and expand Network Adapters in the device display pane.

  2. Right-click the NVIDIA® ConnectX adapter entry and left-click Properties.

  3. Click the Advanced tab and modify the desired properties.

    Warning

    The IPoIB network interface is automatically restarted once you finish modifying IPoIB parameters. Consequently, it might affect any running traffic.

    Important Notes:

    - For help on a specific parameter/option, check the help button at the bottom of the dialog.
    - If you select one of the entries Offload Options, Performance Options, or Flow Control Options, you’ll need to click the Properties button to modify parameters via a pop-up dialog.

RSS settings can be set per individual adapters as well as globally using the Registry Keys below.

Warning

It is recommended that the RSS base processor is core #1 and above as usually processor 0 is very utilized.

For instructions on how to find interface index in registry <nn>, please refer to section Finding the Index Value of the Network Interface.

Sub-key

Description

HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\*MaxRSSProcessors

Maximum number of CPUs allotted. Sets the desired maximum number of processors for each interface. The number can be different for each interface.

Note: Restart the network adapter after you change this registry key.

HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\*RssBaseProcNumber

Base CPU number. Sets the desired base CPU number for each interface. The number can be different for each interface. This allows partitioning of CPUs across network adapters.

Note: Restart the network adapter when you change this registry key.

HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\*NumaNodeID

NUMA node affinitization

HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\*RssBaseProcGroup

Sets the RSS base processor group for systems with more than 64 processors.

HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\RssV2

Enables the RSS V2 feature.

HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\ValidateRssV2

Enable strict argument validation for upper layer testing. Set along with RssV2 key to enable the RSSv2 feature.

Procedure_Heading_Icon.PNG

To display a summary of network adapter software, firmware and hardware related information, perform the following steps:

  1. Display the Device Manager.

    Device_Manager.png

  2. Select the Information tab from the Properties sheet.

    Information_tab.png

    0000018a-66c0-dd68-a38e-f7c3d5d40001

DSCP Sanity Testing

To verify that all QoS and DSCP settings are correct, you can capture the incoming and outgoing traffic by using the mlx5cmd sniffer tool. The tool allows you to see the DSCP value in the captured packets, as displayed in the figure below.

DSCP_values.png


Live Firmware Patch allows ConnectX adapter cards family firmware update (upgrade or downgrade) while the driver, the network ports, and the PCI link remain functional. It is supported only between two firmware versions that support Live Firmware Patch.

To check this capability is available in both firmware versions, burn a new firmware (e.g. using mlxburn) and use “mlxfwreset -d <device> q” command to check if Live Firmware Patch is supported. If Live Firmware Patch feature is supported between two firmware versions, it will be the default reset option.

For example:

Copy
Copied!
            

mlxfwreset -d <device> r

This command will perform Live Firmware Patch if it is possible.

Upon a successful Live Firmware Patch update, the following Event Log message will be generated:

Copy
Copied!
            

<Adapter name>: Firmware version was updated from version %3 to version %4 as a result of the firmware live patch update. Log Name: System Source: mlx5 Event ID: 400 Level: Warning

© Copyright 2023, NVIDIA. Last updated on Sep 5, 2023.