The management utilities described in this chapter are used to manage device’s performance, NIC attributes information and traceability.
The following are the supported management utilities:
mlx5cmd is a general management utility used for configuring the adapter, retrieving its information and collecting its WPP trace.
Usage | mlx5Cmd.exe <tool-name> <tool-arguments> |
Performance Tuning Utility
This utility is used mostly for IP forwarding tests to optimize the driver’s configuration to achieve maximum performance when running in IP router mode.
Usage | mlx5cmd.exe -PerfTuning <tool-arguments> |
Information Utility
This utility displays information of NVIDIA® NIC attributes. It is the equivalent utility to ibstat and vstat utilities in WinOF.
Usage | mlx5cmd.exe -Stat <tool-arguments> |
DriverVersion Utility
The utility can display both the PF's and the VF's driver version.
Usage | mlx5cmd -DriverVersion -hh | -Name <adapter name> | [-PF] | [-VF] <VF number> |
The VF's driver version format naming is different when the VM runs on a Windows or a Linux OS.If the VF number is not set, then all the driver’s VFs’ versions will be printed.
In a VM that runs on Windows OS, the naming format is: Os version,Driver Name,Driver version (e.g., Windows2012R2,WinOF2,2.000.019684)
In a VM that runs on Linux OS, the naming format is: OS,Driver,Driver version
(e.g., Linux Driver: Linux,mlx5_core,4.003.030211; Linux Inbox Driver: Linux,mlx5_core,3.0-1)
Trace Utility
The utility saves the ETW WPP tracing of the driver.
Usage | mlx5cmd.exe -Trace <tool-arguments> |
QoS Configuration Utility
The utility configures Quality of Service (QoS) settings.
Usage | mlx5cmd.exe -QoSConfig -Name <Network Adapter Name> <-DefaultUntaggedPriority | -Dcqcn | -SetupRoceQosConfig> |
For further information about the parameters, you may refer to RCM Configuration.
Quick RoCE Configuration (One-Click RoCE)
This utility provides a quick RoCE configuration method using the mlx5cmd tool. It enables the user to set different QoS RoCE configuration without any pre-requirements.
To set the desired RoCE configuration, run the -Configure <Configuration name> command.
The following are the types of configuration currently support:
Lossy fabric
Lossy fabric with QoS
Lossless fabric
Once set, RoCE will be configured with DSCP priority 26 by default, if the -Priority or -Dscp flags are not specified.
When configuring the interface to work in a "Lossy fabric" state, the configuration is returned to its default (out-of-box) settings and the -Dscp and -Priority flags are ignored.
To check the current configuration, run the -Query command.
Detailed usage | mlx5cmd.exe -QosConfig -SetupRoceQosConfig -h |
Registry Keys Utility
This utility shows the registry keys that were set in the registry and are read by the driver. The PCI information can be queried from the "General" properties tab under "Location".
Usage | mlx5cmd.exe -RegKeys [-bdf <pci-bus#> <pci-device#> <pci-function#>] |
Example | If the "Location" is "PCI Slot 3 (PCI bus 8, device 0, function 0)" mlx5cmd.exe -RegKeys -bdf 8.0.0 |
Non-RSS Traffic Capture Utility
The RssSniffer utility provides sampling of packets that did not pass through the RSS engine, whether it is non-RSS traffic, or in any other case that the hardware determines to avoid RSS hashing.Non-RSS Traffic Capture Utility
The tool generates a packet dump file in a .pcap format. The RSS sampling is performed globally in native RSS mode, or per vPort in virtualization mode, when the hardware vRSS mode is active.
Detailed usage | mlx5cmd.exe -RssSniffer -hh |
Note that the tool can be configured to capture only a part of the packet, as well as specific packets in a sequence (N-th).
Sniffer Utility
Sniffer utility provides the user the ability to capture Ethernet, RoCE and IB traffic that flows to and from the NVIDIA® NIC's ports. The tool generates a packet dump file in .pcap format. This file can be read using the Wireshark tool (www.wireshark.org) for graphical traffic analysis. The .pcap file generated by the Sniffer Utility will be limited by default to 10M. Users can change or cancel the limit size per their demand. In order to force the file limit, the oldest captures will be saved in fileNamePrev.pcap and will be deleted when the limit is reached.
In Bluefield 2 SmartNIC mode, sniffer cannot capture VF to VF traffic.
Detailed usage | mlx5cmd.exe -sniffer -help |
When using the sniffer utility in IPoIB in loopback mode, between VMs and hosts on the same network port, packets are seen twice in the pcap file: once for transmitting and once for receiving.
For multicast packets, packets are seen once for each direction and not for each destination.
The Ethernet Sniffer utility when in SR-IOV mode, on ConnectX-5 and above adapter cards, sniffs only the PF’s traffic and not its VF’s traffic.
Link Speed Utility
This utility provides the ability to query supported link speeds by the adapter. Additionally, it enables the user to force set a particular link speed that the adapter can support.
When using this utility, setting the link speed to 56GbE is not supported.
Usage | mlx5cmd.exe -LinkSpeed -Name <Network Adapter Name> -Query |
Example | mlx5cmd.exe -LinkSpeed -Name <Network Adapter Name> -Set 1 |
Detailed usage | mlx5cmd.exe -LinkSpeed -hh |
Link FEC Configuration Utility
Forward Error Correction (FEC) is an algorithm for finding and fixing errors in data transmission on physical link. The NIC can support several algorithms for every link speed. There is an internal register called PPLM, which contains information on FEC algorithms for every link speed.
PPLM register contains two fields for every link speed - ‘cap’ and ‘admin’.
‘cap’ – means ‘capability’ – is a bitmask field, showing several FEC algorithms, supported for this link speed.
‘admin’ – means ‘configured’ – contains the above ‘cap’ field where only one bit is set. It defines the FEC algorithm which is currently configured.
The Link FEC Configuration utility provides the ability to query supported link FEC modes by the adapter for the current link speed and for all supported link speeds.
Additionally, the utility enables the user to change the default FEC algorithm to one of the FEC modes, that the adapter supports.
Usage | mlx5cmd.exe -Dbg -LinkSpeed -Name <Network Adapter Name> -Query | -QueryPplm | -Set <value> |
Example | mlx5cmd.exe -Dbg -LinkSpeed -Name <Network Adapter Name> -Set RS |
Detailed usage | mlx5cmd.exe -Dbg -LinkSpeed -hh |
NdStat Utility
This utility enumerates open ND connections. Connections can be filtered by adapter IP or Process ID.
Usage | mlx5cmd -NdStat -hh | [-a <IP address>] [-p <Process Id>] [-e] [-n <count>] [-t <time>] |
Example | mlx5cmd -NdStat |
Detailed usage | mlx5cmd -NdStat -hh |
NdkStat Utility
This utility enumerates open NDK connections. Connections can be filtered by adapter IP or Process ID.
Usage | mlx5cmd -NdkStat -hh | [-a <IP address>] [-e] [-n <count>] [-t <time>] |
Example: | mlx5cmd -NdkStat |
Detailed usage | mlx5cmd -NdkStat -hh mlx5cmd -NdkStat -hh |
Debug Utility
This utility exposes driver’s debug information.
Usage | mlx5cmd -Dbg <-PddrInfo | -SwReset> | -hh |
Detailed usage | mlx5cmd -Dbg -hh |
VF Resources
This tool queries VF MSI-X and EQ count.
This tool is not supported in BlueField 2 SmartNIC mode.
Usage | mlx5cmd -Dbg -VfResources -Name <adapter name> |
Detailed usage | mlx5cmd -Dbg -VfResources -hh |
Features Status Utility
The utility displays the status of driver features.
Usage | mlx5cmd -Features -hh | -Name <adapter name> [-Json] [-Indentation <count>] |
Detailed usage | mlx5cmd -Features -hh |
Firmware Capabilities
This tool queries firmware capabilities.
This tool is not supported in BlueField 2 SmartNIC mode.
Usage | mlx5cmd -Dbg -FwCaps -Name <adapter name> mlx5cmd -Dbg -FwCaps -Name <adapter name> -Vf <vf id> mlx5cmd -Dbg -FwCaps -Name <adapter name> -Vf <vf id> -DumpAll |
Detailed usage | mlx5cmd -FwCaps -hh |
Port Diagnostic Database Register (PDDR)
The tool provides troubleshooting and operational information that can assist in debugging physical layer link related issues.
Usage | mlx5cmd -Dbg -PddrInfo [-bdf <pci-bus#> <pci-device#> <pci-function#>] | [-Name <adapter name>] | -hh |
Detailed usage | mlx5cmd -Dbg -PddrInfo -hh |
Software Reset for Adapter Command
The tool enables the user to execute a software reset on the adapter.
Usage | mlx5cmd -Dbg -SwReset -Name <adapter name> |
Detailed usage | mlx5cmd -Dbg -SwReset -hh |
Resource Dump
Resource Dump is used to:
query a menu segments mode:
Usage | mlx5cmd -Dbg -ResourceDump -Menu -hh | -Name <adapter name> |
Detailed usage | mlx5cmd -Dbg -ResourceDump -Menu -hh |
Example | Two menu segment records: mlx5cmd -Dbg -ResourceDump -Menu -Name "Ethernet" ...... ...... __________________________________________________________________ Segment Type - 0x1301 (EQ_BUFF) Dump Params Applicability Special Values -------------------------------- -------------- -------------- index1 -> EQN Mandatory N/A num_of_obj1 N/A N/A index2 -> EQE Optional N/A num_of_obj2 Optional All __________________________________________________________________ __________________________________________________________________ Segment Type - 0x3000 (SX_SLICE) Dump Params Applicability Special Values -------------------------------- -------------- -------------- index1 -> SLICE Mandatory N/A num_of_obj1 N/A N/A index2 -> N/A N/A N/A num_of_obj2 N/A N/A __________________________________________________________________ __________________________________________________________________ ...... ...... |
dump a segments mode:
Usage | mlx5cmd -Dbg -ResourceDump -Menu -hh | -Name <adapter name> |
Detailed usage | mlx5cmd -Dbg -ResourceDump -Menu -hh |
Example | mlx5cmd -Dbg -ResourceDump -Dump -Name "Ethernet" -Segment 0x1310 –Index1 1 Output file generated at C:\Windows\temp\Mlx5_Dump_Me_Now-7-0-0\PF\dmn-GN-OID-RESDUMP-2020.6.17-19.18.16-Gen6 |
The tool does not validate any segment parameters, therefore if any of parameter is missing, the tool will recognize it as zero value. In the case of dump failure, the output file will contain an error message. Hence, we recommend using the menu mode before using this command.
The tool will generate a text file at the printed path, (in our case: “ResourceDump_SegType_0x1310.txt”), and the output text file will contain unparsed text-hex values:
0x0004fffe 0x00000000 0x00000000 0x101b0fb4
0x0005fffa 0x13100000 0x00000001 0x00000000
0x00000000 0x0001fffb
Since the Resource Dump feature is used in DMN to generate a directory, DMN uses a mechanism that limits the number of created directories. For further information, see Cyclic DMN Mechanism.
Packet Pacing Capabilities
This tools query allocated Packet Pacing objects
Usage | mlx5cmd -Dbg -FWPacketPacing -Name <adapter name> |
Detailed usage | mlx5cmd -FWPacketPacing -hh |
Temperature Utility
The tool queries the external ASIC temperature sensor to get temperature readings. It displays the highest temperature among the ASIC diodes on the adapter in Celsius units.
Usage | mlx5cmd -Temperature -hh | [-Name <adapter name>] |
Detailed usage | mlx5cmd -Temperature -hh |
Get-NetView Utility
This utility allows the user to collect data on system and network configurations for troubleshooting purposes.
The utility is only supported on Windows Server 2016 and above. For more information, please refer to the Microsoft SDN repository documentation.
Usage | The script is available publicly as part of the Microsoft repository at 'https://github.com/Microsoft/SDN/blob/master/Diagnostics/Get-NetView.PS1'. To execute the script, simply run the script from PowerShell. Once the script has completed, it will display the output location. |
Display RSS Information
RSS information is now displayed from the driver. On the Hyper-V it will also display Vport's VMMQ configurations.
Usage | mlx5cmd -Dbg -RssInfo -Name <adapter name> [-Json <file_name.json>]| -hh |
smpquery Utility
smpquery allows querying of various information about the InfiniBand network.
Usage | mlx5cmd -ib -SmpQuery -help |
Configuration Validator
This tool validates the configuration of registry keys provided in the configuration file.
Usage | mlx5cmd -ConfigValidator | -Name <Adapter Name> | [-Template] | [-ConfigCompare] | -File <File Name> | -hh |
Detailed usage | mlx5cmd -ConfigValidator -hh |
Example | Print a Template file: mlx5cmd -ConfigValidator -Name cx4 -Template -File .\at.json Compare driver registry configuration with the one in the file: mlx5cmd -Dbg -ConfigValidator -Name cx4 -ConfigCompare -File .\at.json |
VXLAN Offloading Configuration Utility
This tool will allow the user to configure additional ports for VXLAN offloading. The user can also query the VXLAN ports offload configuration of the adapter.
Usage | mlx5cmd -Vxlan -hh | -Name <adapter name> [-add_port <port_num> | -del_port <port_num> | -query] |
Detailed usage | mlx5cmd -Vxlan -hh |
Notes |
|
This feature is supported in NVIDIA® BlueField®-2 devices only. Using the feature on other devices with REAL_TIME timestamping will result in wrong PTP clock.
This feature creates a PTP like ability to let the user sync the clock by getting a PTP similar clock from a DevX commands. To update the system's time, use the value from the “devx_ptp_query_time” option. This feature must be used when the REAL_TIME timestamping is enabled.
The DevX API used for this utility is devx_ptp_create(__deref_out struct devx_ptp_context** ppPtpCtx, __in devx_device_ctx* pDevxCtx, __in uint32_t flags).
To enable the feature:
Create the PTP context (devx_ptp_create).
Query the PTP clock (devx_ptp_query_time).
Repeat the process as many times as needed.
Delete the devx PTP context (devx_ptp_destroy).