Management Utilities

Linux Kernel Upstream Release Notes v6.5

The management utilities described in this chapter are used to manage device’s performance, NIC attributes information and traceability.

The following are the supported management utilities:

mlx5cmd is a general management utility used for configuring the adapter, retrieving its information and collecting its WPP trace.

Usage

mlx5Cmd.exe <tool-name> <tool-arguments>

Performance Tuning Utility

This utility is used mostly for IP forwarding tests to optimize the driver’s configuration to achieve maximum performance when running in IP router mode.

Usage

mlx5cmd.exe -PerfTuning <tool-arguments>

Information Utility

This utility displays information of NVIDIA® NIC attributes. It is the equivalent utility to ibstat and vstat utilities in WinOF.

Usage

mlx5cmd.exe -Stat <tool-arguments>

DriverVersion Utility

The utility can display both the PF's and the VF's driver version.

Usage

mlx5cmd -DriverVersion -hh | -Name <adapter name> | [-PF] | [-VF] <VF number>

The VF's driver version format naming is different when the VM runs on a Windows or a Linux OS.If the VF number is not set, then all the driver’s VFs’ versions will be printed.

  • In a VM that runs on Windows OS, the naming format is: Os version,Driver Name,Driver version (e.g., Windows2012R2,WinOF2,2.000.019684)

  • In a VM that runs on Linux OS, the naming format is: OS,Driver,Driver version

  • (e.g., Linux Driver: Linux,mlx5_core,4.003.030211; Linux Inbox Driver: Linux,mlx5_core,3.0-1)

Trace Utility

The utility saves the ETW WPP tracing of the driver.

Usage

mlx5cmd.exe -Trace <tool-arguments>

QoS Configuration Utility

The utility configures Quality of Service (QoS) settings.

Usage

mlx5cmd.exe -QoSConfig -Name <Network Adapter Name> <-DefaultUntaggedPriority | -Dcqcn | -SetupRoceQosConfig>

For further information about the parameters, you may refer to RCM Configuration.

Quick RoCE Configuration (One-Click RoCE)

This utility provides a quick RoCE configuration method using the mlx5cmd tool. It enables the user to set different QoS RoCE configuration without any pre-requirements.

To set the desired RoCE configuration, run the -Configure <Configuration name> command.

The following are the types of configuration currently support:

  • Lossy fabric

  • Lossy fabric with QoS

  • Lossless fabric

Once set, RoCE will be configured with DSCP priority 26 by default, if the -Priority or -Dscp flags are not specified.

When configuring the interface to work in a "Lossy fabric" state, the configuration is returned to its default (out-of-box) settings and the -Dscp and -Priority flags are ignored.

To check the current configuration, run the -Query command.

Detailed usage

mlx5cmd.exe -QosConfig -SetupRoceQosConfig -h

Registry Keys Utility

This utility shows the registry keys that were set in the registry and are read by the driver. The PCI information can be queried from the "General" properties tab under "Location".

Usage

mlx5cmd.exe -RegKeys [-bdf <pci-bus#> <pci-device#> <pci-function#>]

Example

If the "Location" is "PCI Slot 3 (PCI bus 8, device 0, function 0)"

mlx5cmd.exe -RegKeys -bdf 8.0.0

Non-RSS Traffic Capture Utility

The RssSniffer utility provides sampling of packets that did not pass through the RSS engine, whether it is non-RSS traffic, or in any other case that the hardware determines to avoid RSS hashing.Non-RSS Traffic Capture Utility

The tool generates a packet dump file in a .pcap format. The RSS sampling is performed globally in native RSS mode, or per vPort in virtualization mode, when the hardware vRSS mode is active.

Detailed usage

mlx5cmd.exe -RssSniffer -hh
Warning

Note that the tool can be configured to capture only a part of the packet, as well as specific packets in a sequence (N-th).


Sniffer Utility

Sniffer utility provides the user the ability to capture Ethernet, RoCE and IB traffic that flows to and from the NVIDIA® NIC's ports. The tool generates a packet dump file in .pcap format. This file can be read using the Wireshark tool (www.wireshark.org) for graphical traffic analysis. The .pcap file generated by the Sniffer Utility will be limited by default to 10M. Users can change or cancel the limit size per their demand. In order to force the file limit, the oldest captures will be saved in fileNamePrev.pcap and will be deleted when the limit is reached.

Warning

In Bluefield 2 SmartNIC mode, sniffer cannot capture VF to VF traffic.

Detailed usage

mlx5cmd.exe -sniffer -help
Warning

When using the sniffer utility in IPoIB in loopback mode, between VMs and hosts on the same network port, packets are seen twice in the pcap file: once for transmitting and once for receiving.

For multicast packets, packets are seen once for each direction and not for each destination.

Warning

The Ethernet Sniffer utility when in SR-IOV mode, on ConnectX-5 and above adapter cards, sniffs only the PF’s traffic and not its VF’s traffic.


Link Speed Utility

This utility provides the ability to query supported link speeds by the adapter. Additionally, it enables the user to force set a particular link speed that the adapter can support.

Warning

When using this utility, setting the link speed to 56GbE is not supported.

Usage

mlx5cmd.exe -LinkSpeed -Name <Network Adapter Name> -Query

Example

mlx5cmd.exe -LinkSpeed -Name <Network Adapter Name> -Set 1

Detailed usage

mlx5cmd.exe -LinkSpeed -hh

Link FEC Configuration Utility

Forward Error Correction (FEC) is an algorithm for finding and fixing errors in data transmission on physical link. The NIC can support several algorithms for every link speed. There is an internal register called PPLM, which contains information on FEC algorithms for every link speed.

PPLM register contains two fields for every link speed - ‘cap’ and ‘admin’.

  • ‘cap’ – means ‘capability’ – is a bitmask field, showing several FEC algorithms, supported for this link speed.

  • ‘admin’ – means ‘configured’ – contains the above ‘cap’ field where only one bit is set. It defines the FEC algorithm which is currently configured.

The Link FEC Configuration utility provides the ability to query supported link FEC modes by the adapter for the current link speed and for all supported link speeds.

Additionally, the utility enables the user to change the default FEC algorithm to one of the FEC modes, that the adapter supports.

Usage

mlx5cmd.exe -Dbg -LinkSpeed -Name <Network Adapter Name> -Query | -QueryPplm | -Set <value>

Example

mlx5cmd.exe -Dbg -LinkSpeed -Name <Network Adapter Name> -Set RS

Detailed usage

mlx5cmd.exe -Dbg -LinkSpeed -hh

NdStat Utility

This utility enumerates open ND connections. Connections can be filtered by adapter IP or Process ID.

Usage

mlx5cmd -NdStat -hh | [-a <IP address>] [-p <Process Id>] [-e] [-n <count>] [-t <time>]

Example

mlx5cmd -NdStat

Detailed usage

mlx5cmd -NdStat -hh

NdkStat Utility

This utility enumerates open NDK connections. Connections can be filtered by adapter IP or Process ID.

Usage

mlx5cmd -NdkStat -hh | [-a <IP address>] [-e] [-n <count>] [-t <time>]

Example:

mlx5cmd -NdkStat

Detailed usage

mlx5cmd -NdkStat -hh

mlx5cmd -NdkStat -hh

Debug Utility

This utility exposes driver’s debug information.

Usage

mlx5cmd -Dbg <-PddrInfo | -SwReset> | -hh

Detailed usage

mlx5cmd -Dbg -hh

VF Resources

This tool queries VF MSI-X and EQ count.

Warning

This tool is not supported in BlueField 2 SmartNIC mode.

Usage

mlx5cmd -Dbg -VfResources -Name <adapter name>

mlx5cmd -Dbg -VfResources -Name <adapter name> -Vf <vf id>

Detailed usage

mlx5cmd -Dbg -VfResources -hh

Features Status Utility

The utility displays the status of driver features.

Usage

mlx5cmd -Features -hh | -Name <adapter name> [-Json] [-Indentation <count>]

Detailed usage

mlx5cmd -Features -hh

Firmware Capabilities

This tool queries firmware capabilities.

Warning

This tool is not supported in BlueField 2 SmartNIC mode.

Usage

mlx5cmd -Dbg -FwCaps -Name <adapter name>

mlx5cmd -Dbg -FwCaps -Name <adapter name> -Vf <vf id>

mlx5cmd -Dbg -FwCaps -Name <adapter name> -Vf <vf id> -DumpAll

Detailed usage

mlx5cmd -FwCaps -hh

Port Diagnostic Database Register (PDDR)

The tool provides troubleshooting and operational information that can assist in debugging physical layer link related issues.

Usage

mlx5cmd -Dbg -PddrInfo [-bdf <pci-bus#> <pci-device#> <pci-function#>] | [-Name <adapter name>] | -hh

Detailed usage

mlx5cmd -Dbg -PddrInfo -hh

Software Reset for Adapter Command

The tool enables the user to execute a software reset on the adapter.

Usage

mlx5cmd -Dbg -SwReset -Name <adapter name>

Detailed usage

mlx5cmd -Dbg -SwReset -hh

Resource Dump

Resource Dump is used to:

  • query a menu segments mode:

Usage

mlx5cmd -Dbg -ResourceDump -Menu -hh | -Name <adapter name>

Detailed usage

mlx5cmd -Dbg -ResourceDump -Menu -hh

Example

Two menu segment records:

mlx5cmd -Dbg -ResourceDump -Menu -Name "Ethernet"
......
......
__________________________________________________________________

           Segment Type - 0x1301 (EQ_BUFF)

Dump Params                        Applicability    Special Values
--------------------------------   --------------   --------------
index1 -> EQN                      Mandatory        N/A
num_of_obj1                        N/A              N/A
index2 -> EQE                      Optional         N/A
num_of_obj2                        Optional         All
__________________________________________________________________
__________________________________________________________________

           Segment Type - 0x3000 (SX_SLICE)

Dump Params                        Applicability    Special Values
--------------------------------   --------------   --------------
index1 -> SLICE                    Mandatory        N/A
num_of_obj1                        N/A              N/A
index2 -> N/A                      N/A              N/A
num_of_obj2                        N/A              N/A
__________________________________________________________________
__________________________________________________________________
......
......
  • dump a segments mode:

Usage

mlx5cmd -Dbg -ResourceDump -Menu -hh | -Name <adapter name>

Detailed usage

mlx5cmd -Dbg -ResourceDump -Menu -hh

Example

mlx5cmd -Dbg -ResourceDump -Dump -Name "Ethernet" -Segment 0x1310 –Index1 1
Output file generated at C:\Windows\temp\Mlx5_Dump_Me_Now-7-0-0\PF\dmn-GN-OID-RESDUMP-2020.6.17-19.18.16-Gen6
Warning

The tool does not validate any segment parameters, therefore if any of parameter is missing, the tool will recognize it as zero value. In the case of dump failure, the output file will contain an error message. Hence, we recommend using the menu mode before using this command.

The tool will generate a text file at the printed path, (in our case: “ResourceDump_SegType_0x1310.txt”), and the output text file will contain unparsed text-hex values:

Copy
Copied!
            

0x0004fffe 0x00000000 0x00000000 0x101b0fb4 0x0005fffa 0x13100000 0x00000001 0x00000000 0x00000000 0x0001fffb

Warning

Since the Resource Dump feature is used in DMN to generate a directory, DMN uses a mechanism that limits the number of created directories. For further information, see Cyclic DMN Mechanism.


Packet Pacing Capabilities

This tools query allocated Packet Pacing objects

Usage

mlx5cmd -Dbg -FWPacketPacing -Name <adapter name>

mlx5cmd -Dbg -FWPacketPacing -Name <adapter name> -Index <index id>

mlx5cmd -Dbg -FWPacketPacing -Name <adapter name> -UID <uid>

Detailed usage

mlx5cmd -FWPacketPacing -hh

Temperature Utility

The tool queries the external ASIC temperature sensor to get temperature readings. It displays the highest temperature among the ASIC diodes on the adapter in Celsius units.

Usage

mlx5cmd -Temperature -hh | [-Name <adapter name>]

Detailed usage

mlx5cmd -Temperature -hh

Get-NetView Utility

This utility allows the user to collect data on system and network configurations for troubleshooting purposes.

Warning

The utility is only supported on Windows Server 2016 and above. For more information, please refer to the Microsoft SDN repository documentation.

Usage

The script is available publicly as part of the Microsoft repository at 'https://github.com/Microsoft/SDN/blob/master/Diagnostics/Get-NetView.PS1'.

To execute the script, simply run the script from PowerShell. Once the script has completed, it will display the output location.


Display RSS Information

RSS information is now displayed from the driver. On the Hyper-V it will also display Vport's VMMQ configurations.

Usage

mlx5cmd -Dbg -RssInfo -Name <adapter name> [-Json <file_name.json>]| -hh

smpquery Utility

smpquery allows querying of various information about the InfiniBand network.

Usage

mlx5cmd -ib -SmpQuery -help

Configuration Validator

This tool validates the configuration of registry keys provided in the configuration file.

Usage

mlx5cmd -ConfigValidator | -Name <Adapter Name> | [-Template] | [-ConfigCompare] | -File <File Name> | -hh

Detailed usage

mlx5cmd -ConfigValidator -hh

Example

Print a Template file:

mlx5cmd -ConfigValidator -Name cx4 -Template -File .\at.json

Compare driver registry configuration with the one in the file:

mlx5cmd -Dbg -ConfigValidator -Name cx4 -ConfigCompare -File .\at.json

VXLAN Offloading Configuration Utility

This tool will allow the user to configure additional ports for VXLAN offloading. The user can also query the VXLAN ports offload configuration of the adapter.

Usage

mlx5cmd -Vxlan -hh | -Name <adapter name> [-add_port <port_num> | -del_port <port_num> | -query]

Detailed usage

mlx5cmd -Vxlan -hh

Notes

  • VXLAN offloading is a global hardware configuration, therefore any modification applies to all adapter ports.

  • VXLAN offloading is always configured on the IANA standard VXLAN port, regardless of OS configuration.


The AutoLogger is a debuggability capability implemented as part of Mlx5Cmd, that automatically collects logs until it detects a trigger defined by the user.

Usage

mlx5cmd -AutoLogger -hh | [-Name <adapter name>] -TriggerType <type>

Detailed usage

mlx5cmd -AutoLogger -hh
Warning

This feature is supported in NVIDIA® BlueField®-2 devices only. Using the feature on other devices with REAL_TIME timestamping will result in wrong PTP clock.

This feature creates a PTP like ability to let the user sync the clock by getting a PTP similar clock from a DevX commands. To update the system's time, use the value from the “devx_ptp_query_time” option. This feature must be used when the REAL_TIME timestamping is enabled.
The DevX API used for this utility is devx_ptp_create(__deref_out struct devx_ptp_context** ppPtpCtx, __in devx_device_ctx* pDevxCtx, __in uint32_t flags).

To enable the feature:

  1. Create the PTP context (devx_ptp_create).

  2. Query the PTP clock (devx_ptp_query_time).

  3. Repeat the process as many times as needed.

  4. Delete the devx PTP context (devx_ptp_destroy).

© Copyright 2023, NVIDIA. Last updated on Nov 1, 2023.