RSS Monitoring

The RSS monitoring feature allows users to track the packets received by the Mellanox adapters on their machine.
The feature is comprised of two parts:

• Non-RSS Packet Capturing

  • A command-line interface for capturing received non-RSS packets.

• Packet monitoring through the performance monitor

  • A counter in the performance monitor for tracking the received non-RSS packets

Both parts of the feature must be enabled (per adapter instance), using the registry key: "RssMonitoringEnabled". Restart the adapter for these changes to take effect.

Registry key location:

HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\RssMonitoringEnabled

For instructions on how to identify the interface index of a network interface in the registry <nn>, please refer to Finding the Index Value of the Network Interface.

Controlling the RSS Monitoring Registry Key Feature

This feature is added as a flag to "mlxtool.exe", which is installed as part of the WinOF package. The tool can be used to capture the Non-RSS packets received by the machine, and dump the captured traffic into a PCAP file.

Tool Default Path:

C:\Program Files\Mellanox\MLNX_VPI\Tools\mlxtool.exe

Using the Tool

Step 1. Enable the feature through the registry, as described in RSS Monitoring above.
Step 2. Obtain the name of the Mellanox adapter on your machine.

            

            
> mlxtool.exe show ports
        

Step 3. Query the status of the packet capturing. This can be used to tell if the tool is currently running:

            

            
> mlxtool.exe dbg nonrss-capture "Mellanox Adapter Name" query
        

Step 4. Start the packet capturing:

• When activating the feature, the user can specify two parameters:

  • Circular Buffer Size>: The number of packets to store in the capture buffer. This buffer is flushed when "collect" is called (see #5 for an description of collect). This is a circular buffer, so once it is full, it will override previous entries. If not specified, the default will be used. Default: 1024 packets.

  • <Number of Bytes to Capture per Packet>: The number of bytes to capture from each packet. Users can choose to only capture a "snip" of each packet. If not specified, the default will be used. Default: 128 bytes.

• In case of VPort mode (when NicSwitch is enabled), these parameters are global for all VPorts. The feature is either enabled on all VPorts or disabled on all VPorts. The circular buffer size and number of bytes to capture per packet cannot be specified per VPort.

            

            
> mlxtool.exe dbg nonrss-capture "Mellanox Adapter Name" start <Circular Buffer Size> <Number of Bytes to Capture per Packet>
        

Step 5. Collect the captured packets:

• Packets are stored in a circular buffer. They are dumped to a PCAP file when "collect" is called. This must be called before calling "stop", since the buffer gets cleared once the tool is stopped. The status of the feature can be retrieved using the "query" command described in #3.

• When capturing the dump, the user can specify two parameters:

• <PCAP file name>: The name of the packet dump file. If not specified, the default will be used. Default: nonrss.pcap.

• <VPort ID>: The Vport ID to collect for. This is only valid when the driver is operating in VPort mode (NicSwitch is enabled). If not specified, the default will be used. Default: 0.

            

            
> mlxtool.exe dbg nonrss-capture "Mellanox Adapter Name" collect <PCAP File Name> <VPort ID>
        

Step 6. Stop the packet capture.

• If "collect" is not called before the packet capture is stopped, the buffer that contains the captured packets will be cleared. Call "collect" before stopping the feature to dump the packets to a PCAP file.

            

            
> mlxtool.exe dbg nonrss-capture "Mellanox Adapter Name" stop
        

Packet Monitoring through the Performance Monitor

This feature activates counters in the performance monitor which can be used to monitor the packets that are being received by the Mellanox adapter(s). The feature allows the user to monitor incoming traffic for each adapter. It is split up per VPort (if NicSwitch is enabled) and per CPU.

All available counters are listed in the following figure:

Using the Tool

Step 1. Enable the feature through the registry, as described in RSS Monitoring above. In addition to the "RssMonitoringEnabled" registry key, this feature has an additional registry key "RssCountersActivatedAtStartup". This counter will begin counting received packets as soon as the adapter is connected. If this is not set, the counter will begin counting received packets as soon as the counter is added to the performance monitor. Restart the adapter for these changes to take effect.

Registry key location:

HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\RssCountersActivatedAtStartup

For instructions on how to identify the interface index of a network interface in the registry <nn>, please refer to Finding the Index Value of the Network Interface.

Disable and Enable RSS Monitoring On Adapter Startup with the RssCountersActivatedAtStartup Registry Key

Step 2. Open up the Windows performance monitor and add a new counter. The counter will be called "Mellanox Adapter Rss Counters".

Step 3. Select and add the desired instances of the "Mellanox Adapter Rss Counters".

• Note: There will be instances for each Mellanox Adapter that has the proper registry keys set (see introduction to Rss Monitoring for more information on registry keys).

• Note: In VPort mode (when NicSwitch is enabled), there will be a counter available per VPort for each CPU. There is also a "Total" for each VPort, which will report the total packets received by that VPort.
  In native mode, there will be a counter available per CPU. There will also be a "Total" for each adapter, which will report the total packets received by that adapter.

Step 4. Switch the graph type in the performance monitor to "Report".

Step 5. Once the counters are added, the counters will get incremented based on the traffic that is received.

The Proprietary Adapter Mellanox RSS counter set consists of software counters per vport (in case of native RSS it is the physical port) and per CPU. These counters collect information about the RSS and non-RSS traffic received on a specific vport (or physical port for native RSS) and on a specific CPU. The counter set also defines a "Total" instance to collect on all CPUs for each available vport.

Controlling the Counters

As of Version 5.30, the Adapter RSS Counter set is disabled by default. In order to enable it, the following registry value must be set to "1":
HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\RssMonitoringEnabled

Since these are software counters, and may have impact on the performance, they are activated only when the user has explicitly added the desired counters instance in Perfmon (or any other PCW application).

If "always on" behavior is desired (start counting upon driver load), the following registry key must be set to "1":
HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<nn>\RssCountersActivatedAtStartup

Proprietary RDMA Activity

Proprietary RDMA Activity counter set consists of NDK and NDSPI performance counters. These performance counters allow you to track Network Direct Kernel (RDMA) activity, including traffic rates, errors, and control plane activity.

Note a. These counters are only implemented in NDK and are not implemented in NDSPI.