RXP Bench

NVIDIA DOCA RXP Bench User Guide

RXP Bench is a performance comparison tool for NVIDIA® BlueField®-2 RXP.

RXP Bench is a tool that allows for the performance comparison between the NVIDIA® RXP® hardware RegEx acceleration engine found in the NVIDIA® BlueField®-2 DPU and the Intel® Hyperscan software library. It provides a comprehensive set of options and can facilitate ingress of data from live network ports or previously recorded PCAP files.

It is designed to provide a real-world comparison of these technologies, and present results customers could expect to receive after implementing either technologies within their products.

1.1. Scope

This document provides the following information for RXP Bench:

• Example use case
• Breakdown of analysis and runtime statistics
• Options and configuration settings available

1.2. Glossary

The terms listed in the following table are used in this document.

The acronyms listed in the following table are used in this document.

1.3. Icons

The following icons are used within this document:

• 

  – the configuration option is related to a physical DPDK port

• 

  – the configuration option is related to a PCAP format file

• 

  – the configuraiton option is related to a standard text file

Whilst the primary focus of this tool is to provide accurate real-world performance comparisons between the Intel® Hyperscan software library (HS) and the BlueField-2 RXP hardware acceleration engine has additional functionality. This functionality includes:

• Execution on both Intel host and the BlueField-2 DPU Arm cores
• Multicore support
• Ingress of traffic from live DPDK network ports, or PCAP files
• Can act as a "bump in the wire"
• Ability to accept RXP, Hyperscan, and generic rules files
• Asynchronous operations, similar to end-user applications
• Comprehensive configuration through a configuration file or command line options
• A high-performance reference application for DPDK RegEx operations

RXP Bench utilizes the DPDK framework to provide both packet operations and the hardware accelerated Regular Expression offloading (dpdk_regex). Hyperscan is provided through the HS.

2.1. Host Installation

RXP Bench is included as part of the DOCA installation. Separate installation packages are provided for Ubuntu 18.04 and Ubuntu 20.04, whilst CentOS 7.6, 8.0 and 8.2 are supported through a single package:

2.1.1. Prerequisites

Prior to execution of the RXP Bench, an installation of Hyperscan must be present on the host. Hyperscan can be obtained from your Linux distributions package manager (APT, dpkg, yum, etc.) or alternatively compiled from the source. Depending on your installation the following version of Hyperscan is required:

            

            
apt install libhyperscan4
        

            

            
apt install libhyperscan5
        

            

            
yum install epel-release sudo yum install http://repo.openfusion.net/centos7-x86_64/hyperscan-5.3.0-1.of.el7.x86_64.rpm
        

            

            
yum install epel-release
sudo yum install https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/h/hyperscan-5.3.0-5.el8.x86_64.rpm
        

2.2. DPU Installation

The RXP Bench utility is provided as part of the DOCA framework and is therefore installed by default with the BFB.

This section details an example use case of the RXP Bench application, providing in depth explanations of the processes and statistics produced.

This example will focus on the execution of RXP Bench using a simple text file containing the works of William Shakespeare (shakespeare.txt), using rules in Hyperscan format (henry.hs) whilst executing on the BlueField-2 RXP engine.

3.1. Configuring RXP Bench

RXP Bench supports the configuration of options through a “configuration” file, or through the command line. In practice if a configure file is used, the command line options are still available and will override any options already present in the configuration file.

By default, RXP Bench will always search for a “rxpbench.conf”, this allows common set-up commands to be removed from the command line. Commonly the DPDK EAL (-D) options are placed in this file as they rarely change after being initially set.

For a full list of options please see section General Configuration Options.

In this example we are providing all the options on the command-line; there is no configuration file. The command-line required to execute our example (simple text file containing the works of William Shakespeare (shakespeare.txt), using rules in Hyperscan format (henry.hs) whilst executing on the BlueField-2 RXP engine) is as follows:

            

            
./rxpbench -D “-l 0,1,2,3 -n 1 -a 5e:00.0,class=regex –file-prefix=rxpbench -a 5e:00.01” --input-mode text_file -f ../Shakespeare.txt -d rxp -R ../henry.hs -l 2048 -n 10000 -c 1
        

The -D option provides the DPDK EAL options, contained within a set of quotation marks (“). These options are passed directly to DPDK during the initialization of the application and are in general specific to your host. "

The first RXP Bench option is the “--input-mode” which states that RXP Bench will pull data from a “text_file”, the “-f” option then specifies the location and name of the text file to be searched.

The “-d” option states the mode in which RXP Bench will operate, available options are “rxp” or “hs” and in this instance we are requesting that the BlueField-2 hardware accelerator is used.

The "-R" option provides the tool with a set of uncompiled rules, in this case they are presented in Hyperscan format. RXP Bench supports the use of rules formats that are different from the selected device / algorithm. For example, the RXP can accept Hyperscan rules and the Hyperscan library can received RXP formatted rules. The conversion process within RXP Bench is automatic.

The “-l” option supplies the size of the data block sent to the device / algorithm. In this instance a buffer of 2KB is received and pattern matched from the text file.

The number of iterations is controlled by the “-n” option; due to the high performance of the BlueField-2 RXP engine the input file must be iterated 10,000 times to provide enough input data to ensure are run-time of a few seconds.

The final option is the core count (-c), this defines how many CPU cores the tool can use. In this instance we are using a single core.

3.2. Regular Expressions

RXP Bench accept regular expressions in two different formats:

• Uncompiled – The regular expressions are presented in a text file which follows with the RXP rules file format, or the Hyperscan file format.
• Compiled – In the case for the RXP, rules are externally compiled using the RXP Compiler (rxpc) and presented to RXP Bench as a ROF file.

If uncompiled rule files are used, RXP Bench can cross compile the rules regardless of the file format or device selected, i.e. A Hyperscan format rules file will be converted for use by the RXP engine, whilst an RXP format rules file will be converted for use by the Hyperscan engine.

In this example we have requested the Hyperscan rules be used on the RXP engine; in this instance you will see the rxpc compiler being invoked by RXP Bench to provide the on-demand compilation of the rules:

After this process is complete, the tool will automatically program the compiled rules into the hardware and begin the process of Regex pattern matching.

3.3. Runtime Statistics

During the execution of RXP Bench a series of run-time statistics are presented by the utility. This provides detailed information on the current process:

For each core in use by the tool, the following statistic are presented:

• Received Bytes – These are bytes received from the input source
• Regex Bytes – These are the bytes transmitted to the Regex engine; this value can be less than the received byte count if certain confirmation options are used (such as payload thresholds or if “app layer” payloads are only being scanned)
• Recv Bufs – The total buffers received from the input source; in this case due to the “-l 2048” option each buffer contains 2048 bytes.
• Regex Bugfs – The number of buffers transmitted to the Regex device.
• Matches – The total number of Regex matches seen in the input data.

  The total number of Regex matches seen in the input data.

In addition to each core statistic, a running total output is provided, including aggregated values for the above fields and a duration field it also provides:

• Regex Perf (total) – The performance total in Gigabits per second (Gb/s) for the entire duration of the run.
• Regex Perf (split) – The performance total in Gigabits per second (Gb/s) for the past update period.

3.4. End-of-Run Statistics

When the execution is completed, or aborted using Ctrl + C, several statistics blocks are output to the console. This allows users to verify and understand the execution of the performance test.

3.4.1. Configuration Statistics Block

This section of statistics provides an overview of the RXP Bench configuration, most of this information is simply the mirroring of configuration files.

The “Preloaded Data Info” section details any preloading of data, when using PCAP or text files, that has occurred during the initialization of the application:

• Data Length – When the input is file based (PCAP or Text) this is the total data that is preloaded/cached to reduce I/O operations
• App Layer Mode – Whether the application is effectively scanning the application layer (TCP/UDP frames) and ignore the headers (Ethernet, MAC, etc.) prior to the application layer.
• Valid Packets – If app layer mode is enabled, these are packets that contain a valid payload
• Invalid Length – This value is incremented if a PCAP packet is found to be unexpectantly truncated
• Unsupported Prot – If app layer mode is enabled, the packet did not contain one of the required protocols (VLAN/IPv4/IPv6/TCP or UDP)

3.4.2. Run Overview Block

This section provides an overview of the RXP Bench execution; it provides the core statistics which allow you to gauge the performance of the algorithm using the supplied rules and input data.

Whilst most of these fields are self-explanatory some fields require further definition:

• Packet Processing Rate (Mpps) – This is the rate which, in million packets per second, RXP Bench has been able to acquire packets from the input source (Physical port or precached PCAP/text file). For the physical ports this rate may be different that the RegEx PPR value as not all packets (depending on configuration) may be sent to the Regex device.
• Packet Processing Perf (Gb/s) – The actual data-rate of the input source in Gigabits per second
• Total Regex Buffers – This is the number of complete buffers that were sent to the RegEx device for processing
• Total Regex Bytes – The total bytes contained within all buffers transmitted to the RegEx device for processing
• Total Regex Batches – RegEx buffers are gathered together into batches (based on the "-g" flag) and submitted to the RegEx device in a single operation

3.4.3. DPDK RegEx Stats Block

If the selected RegEx device is "rxp" or "regex_dpdk" the following block of statistics is provided. It presents more internal statistics from the DPDK RegEx device (BlueField-2 RXP):

The following are the definitions of each of these counters:

• Invalid Responses – These are responses from operations that have not completed successfully
• Timeout – When processing a block of input data a hardware triggered timeout occurred and the search was aborted
• Max Matches – The maximum number of configured matches was exceeded, and the job was aborted
• Max Prefixes – The maximum prefixes per scan was exceeded, and the job was aborted
• Resource Limit – A generic/internal resourcing limit was reached; the job was aborted
• Latency Figures – These provides max/min and average latency of jobs from transmitted to the DPDK RegEx device

3.4.4. Hyperscan Stats Block

If the selected Regex device is “hs” (or “Hyperscan”) then an additional block of statistics is provided detailing the latency of requests to and response from the Hyperscan Library:

Configuration options to control the operation of RXP Bench can be provided either through a pre-defined configuration file or through the command line. If both a configuration file is supplied and a set of command line options then the command line options will supersede, effectively overriding, the options present in the configuration file.

4.1. Configuration File (-C, --config-file)

The configuration file option allows you to supply a text file that contains one or more options that would normally be present on the command line.

            

            
-C configuration.file
--config-file configuration.file
        

The file should contain each configuration option stripped of the leading dashes on a new line. A colon (:) should be placed between the option and the value. You may use either the short (-) or long (--) option name. For example:

            

            
input-mode : dpdk_port
m : inputfile.pcap
run-time-secs : 10
        

If the “-C” or “--config-file” option is used without any supplied parameter, RXP Bench will attempt to open the default file “rxpbench.conf”.

4.2. DPDK EAL (-D)

RXP Bench utilizes the DPDK framework to provide core memory management, packet ingress and Regular expression offloading. As common with DPDK applications there are several EAL options that can be used to ensure DPDK is optimally configured for the host environment.

EAL options should be enclosed in quotations (“..”) and are passed directly to DPDK without any processing by RXP Bench.

Please ensure if you’re are created a custom set of EAL commands that the “class=regex” parameter is included to ensure the Regex devices is available for use. You should use the “class=eth:regex” if you wish to use packet acquisition from physical ports and Regex.

4.3. Force Compilation (-F)

RXP Bench can accept both uncompiled and compiled rules. As part of the initialization process any uncompiled regular expressions rules must be compiled into object code that can be executed on the BF2 RXP hardware accelerator, or Intel™ Hyperscan software library.

Whilst the BlueField-2 RXP supports a wide range of Regular Expression constructs, both itself and Hyperscan cannot provide for all constructs due to complexity and performance impacts.

When a supplied set of Regular expressions is compiled either algorithm may abort the compilation due to the inclusion of (one or more) unsupported rule constructs. This option prevents the compilers from aborting, and forces RXP Bench to continue with the rules that did successful compile.

4.4. Verbose (-V)

This option provides additional verbose output on any matching patterns found by the Regex algorithm. The supplied integer value dictates the amount of information provided:

            

            
-V 1
-V 2
        

Both verbose levels' will write out to a CSV files named "rxpbench_matches_main_core_XX.csv", where XX is the main logical core ID returned by DPDK, and "rxpbench_matches_core_XX.csv" for additional cores in a multicore environment.

Each entry in the CSV file provides match information including queue ID, rule ID, start offset and length. If the verbose level is set to “2” then the match string is also returned.

4.5. Cores (-c)

The "Cores" option allows for the configuration of the total number of cores available to RXP Bench.

            

            
-c 4
        

The use of the CPU cores is dependent on the application’s Regular Expression algorithm and whether packets are being received from an ingress port or PCAP capture file.

If the BlueField-2 RXP hardware accelerator is used each core will be given a unique DPDK Regex queue to operate on; if the accelerator is Hyperscan then each core will be used to execute the Hyperscan software library.

In addition, if packets are being received from a physical port, the value will be used to allocate X number of DPDK Tx and Rx queues on the port.

This group of options provides the ability to select the Algorithm (BF2 RXP or Hyperscan), where input data should be received from (physical ports, text files, or PCAP files) and Regular Expression rules information.

5.1. Algorithm/Device Select (--Regex-dev, -d)

The algorithm or device to be used is supplied through this option. The available options are "regex_dpdk"/"rxp" for the BlueField-2 RXP Hardware accelerator, or "hyperscan"/"hs" for the Intel Hyperscan software library.

            

            
--Regex-dev regex_dpdk 
--Regex-dev rxp
--Regex-dev hyperscan
--Regex-dev hs
        

5.2. Input Mode (--input-mode, -m)

RXP Bench can receive data from various input sources. This option allows you to provide which method you require:

            

            
--input-mode dpdk_port --dpdp-primary-port X --dpdk-secondary-port Y
--input-mode pcap_file
--input-mode text_file
        

--input-mode dpdk_port, -m dpdk_port

The DPDK port option enables RXP Bench to receive live traffic from a port, specified in the --dpdk-primary-port. If the secondary port option exists (--dpdk-second-port) then any packets received, after pattern matching has occurred, are transmitted onto the second port.

See section DPDK Port Operations for more information.

--input-mode pcap_file, -m pcap_file

This option allows you to supply an external PCAP file. This allows for reproducible results using a known input file. The entire payload recorded in each frame within the pcap file is made available to RXP Bench.

--input-mode text_file, -m text_file

If processing of a standard text file is required, this option allows you to select any file. The entire text file contents are made available to the RXP Bench application with no parsing or changes made.

5.3. Compiled Rules File (--rules, -r)

The RXP hardware accelerator can accept regular expressions that have been externally compiled using the RXPC (RXP Compiler) into a ROFF file. This option allows you to specify this ROF2 file.

5.4. Uncompiled Rules File (--raw_rules, -R)

RXP Bench can accept an input file containing raw regular expressions. The uncompiled rules file can be in either of these formats:

• RXP rules file
• Hyperscan rules file

The tool can accept either format of rules file, regardless of which algorithm (BlueField-2 RXP, or Intel Hyperscan) is used. In the case where a rules file is not in the expected format for the algorithm, a conversion process is employed to ensure they operate correctly.

When input data is received from a physical network port, the options in the subsequent sections can be used to configure the ports.

6.1. Primary Port (--dpdk-primary-port, -1)

This is the port where packets will be received from. The supplied ID is used directly to access the requested DPDK port.

6.2. Secondary Port (--dpdk-secondary-port, -2)

RXP Bench can be used as a "bump" in the wire, where received packets are pattern matched before transmission through a secondary port. This option provides the port ID, as used by DPDK, for the onwards transmission of scanned packets.

This groups of options allows for the specific duration of any execution to be controlled through various metrics.

7.1. Runtime Seconds (--run-time-secs, -s)

In live packet acquisition, i.e. packets are received from a physics port, this option controls the number of seconds that packet ingress and pattern matching occur for.

7.2. Iterations (--run-num-iterations, -n)

If input data is being received from either a PCAP File or text file, this option is used to limit the execute to a complete number of iterations of the input file.

For example, if an iteration count of 4 was given on a PCAP file contains 1,000 packets. The total number of packets processed would be 4,000. If the input file was a standard text file containing 5,000 bytes of information, an iteration count of 4 would mean 20,000 bytes would be read by RXP Bench.

7.3. Packet (--run-packets, -p)

For both live traffic reception and PCAP input files, this option limits the total execution to the supplied number of packets.

7.4. Total Bytes (--run-bytes, -b)

Regardless of the input mode this is the total number of bytes received that is required to mark the execution as complete.

For live traffic this is the total number of bytes received from the physical port. For both the PCAP input file and text file this is the total bytes to read from the input files.

These options allow for finer control of data to be transmitted to the RegEx device for pattern matching.

8.1. Buffer Length (--buf-length, -l)

When the RXP Bench is reading from input files (whether PCAP, or text files) it has all the information readily available (unlike live traffic which must be received). This allows the application to read a variable amount of input data per iteration.

This option controls the amount of data that is read from the input file and passed to the Regular Expression algorithm.

8.2. Buffer Threshold (--buf-thres, -t)

When live traffic is being received from a physical port, this option specifies the received packets minimum size before it will be processed.

For example, setting this value to 256 bytes means that if a packet arrives that is less than 256 bytes in length it will not be processed by RXP Bench.

Packets that are dropped by this threshold are recorded in the statistics under “UNDER THRES” field.

8.3. Buffer Overlapping (--buf-overlap, -o)

When the input is being read from files (either PCAP, or text files) this option allows a certain number of bytes to be overlapped from the previous frame.

8.4. Batching (--buf-group, -g)

Most high-performance applications obtain additional performance by batching together multiple operations into a single process.

DPDK Regex provides the capability of enqueuing multiple buffers to the BlueField-2 RXP Hardware accelerator. This option allows you to specify how many payloads should be grouped together before enqueuing on the hardware.

If receiving packets from a physical port this also determines the batch size to read (and write) to the network ports.

Several options that are specific to the RXP hardware accelerator exist. These allow you to modify the behavior of the Regular Expression pattern matching engine.

9.1. Max Matches (--rxp-max-matches, -M)

The RXP engine provides a method to alter the maximum number of matches reported by any operation; by default, this value is set to its maximum value of 255.

Altering this value will result in the engine completing its processing of the data as soon as the maximum matches is found.

9.2. Max Latency (--rxp-latency, -T)

Like all algorithms, the RXP engine takes a certain amount of time to complete its scanning of data for regular expression matches.

This option can alter the maximum time that the hardware will execute any given job for. After this time has been exceeded the job will be aborted.

Several options that are specific to the Hyperscan software library exist. These allow you to modify the behavior of the Regular Expression pattern matching engine.

HS Single Match (--hs-singlematch, -H)

The Hyperscan algorithm provides an option called "HS_FLAG_SINGLEMATCH", please see the Hyperscan documentation for more information.

HS Left Most Match (--hs- leftmost, -L)

The Hyperscan algorithm provides an option called "HS_FLAG_SOM_LEFTMOST", please see the Hyperscan documentation for more information.

Notice

This document is provided for information purposes only and shall not be regarded as a warranty of a certain functionality, condition, or quality of a product. NVIDIA Corporation nor any of its direct or indirect subsidiaries (collectively: “NVIDIA”) make no representations or warranties, expressed or implied, as to the accuracy or completeness of the information contained in this document and assume no responsibility for any errors contained herein. NVIDIA shall have no liability for the consequences or use of such information or for any infringement of patents or other rights of third parties that may result from its use. This document is not a commitment to develop, release, or deliver any Material (defined below), code, or functionality.

NVIDIA reserves the right to make corrections, modifications, enhancements, improvements, and any other changes to this document, at any time without notice.

Customer should obtain the latest relevant information before placing orders and should verify that such information is current and complete.

NVIDIA products are sold subject to the NVIDIA standard terms and conditions of sale supplied at the time of order acknowledgement, unless otherwise agreed in an individual sales agreement signed by authorized representatives of NVIDIA and customer (“Terms of Sale”). NVIDIA hereby expressly objects to applying any customer general terms and conditions with regards to the purchase of the NVIDIA product referenced in this document. No contractual obligations are formed either directly or indirectly by this document.

NVIDIA products are not designed, authorized, or warranted to be suitable for use in medical, military, aircraft, space, or life support equipment, nor in applications where failure or malfunction of the NVIDIA product can reasonably be expected to result in personal injury, death, or property or environmental damage. NVIDIA accepts no liability for inclusion and/or use of NVIDIA products in such equipment or applications and therefore such inclusion and/or use is at customer’s own risk.

NVIDIA makes no representation or warranty that products based on this document will be suitable for any specified use. Testing of all parameters of each product is not necessarily performed by NVIDIA. It is customer’s sole responsibility to evaluate and determine the applicability of any information contained in this document, ensure the product is suitable and fit for the application planned by customer, and perform the necessary testing for the application in order to avoid a default of the application or the product. Weaknesses in customer’s product designs may affect the quality and reliability of the NVIDIA product and may result in additional or different conditions and/or requirements beyond those contained in this document. NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs.

No license, either expressed or implied, is granted under any NVIDIA patent right, copyright, or other NVIDIA intellectual property right under this document. Information published by NVIDIA regarding third-party products or services does not constitute a license from NVIDIA to use such products or services or a warranty or endorsement thereof. Use of such information may require a license from a third party under the patents or other intellectual property rights of the third party, or a license from NVIDIA under the patents or other intellectual property rights of NVIDIA.

Reproduction of information in this document is permissible only if approved in advance by NVIDIA in writing, reproduced without alteration and in full compliance with all applicable export laws and regulations, and accompanied by all associated conditions, limitations, and notices.

THIS DOCUMENT AND ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, “MATERIALS”) ARE BEING PROVIDED “AS IS.” NVIDIA MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE. TO THE EXTENT NOT PROHIBITED BY LAW, IN NO EVENT WILL NVIDIA BE LIABLE FOR ANY DAMAGES, INCLUDING WITHOUT LIMITATION ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES, HOWEVER CAUSED AND REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF ANY USE OF THIS DOCUMENT, EVEN IF NVIDIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Notwithstanding any damages that customer might incur for any reason whatsoever, NVIDIA’s aggregate and cumulative liability towards customer for the products described herein shall be limited in accordance with the Terms of Sale for the product.

Trademarks

NVIDIA, the NVIDIA logo, and Mellanox are trademarks and/or registered trademarks of Mellanox Technologies Ltd. and/or NVIDIA Corporation in the U.S. and in other countries. Other company and product names may be trademarks of the respective companies with which they are associated.

Copyright

© 2021 NVIDIA Corporation. All rights reserved.