QSG for Configuring TRex in a few steps using Nvidia ConnectX adapters

Created on Jul 11, 2019

image2019-7-11_13-52-54.png

TRex is an open source stateful and stateless traffic generator based on DPDK.

With single ConnectX ethernet card family it can generate 100Gbs bandwidth easily.

The more cores are used the more Mpps rate will increase significantly allowing more than 100Mpps!

This QSG (Quick Start Guide) article will guide a user for quick and powerful installation.

References

Prerequisite

1. Install Centos 8.2 operating system.

2. Install rdma-core:

CLI Commands

Copy
Copied!
            

yum install rdma-core-devel

Installation

1. Download and extract TRex latest branch.

CLI Commands

Copy
Copied!
            

wget --no-check-certificate https://trex-tgn.cisco.com/trex/release/latest tar -xzvf latest

2. Run initialization script

CLI Commands

Copy
Copied!
            

cd /<latest ver_name> ./dpdk_setup_ports.py -i   By default, IP based configuration file will be created. Do you want to use MAC based config? (y/N) n   * y will define physical MAC address per port in the configuration file. N will keep default IP per port identifier.   +----+------+---------+-------------------+-----------------------------------------+-----------+----------+----------+   | ID | NUMA | PCI | MAC | Name | Driver | Linux IF | Active |   +====+======+=========+===================+=========================================+===========+==========+==========+   | 0 | 0 | 03:00.0 | a0:d3:c1:01:43:ac | NetXtreme BCM5719 Gigabit Ethernet PCIe | tg3 | eno1 | *Active* |   +----+------+---------+-------------------+-----------------------------------------+-----------+----------+----------+   | 1 | 0 | 03:00.1 | a0:d3:c1:01:43:ad | NetXtreme BCM5719 Gigabit Ethernet PCIe | tg3 | eno2 | |   +----+------+---------+-------------------+-----------------------------------------+-----------+----------+----------+   | 2 | 0 | 03:00.2 | a0:d3:c1:01:43:ae | NetXtreme BCM5719 Gigabit Ethernet PCIe | tg3 | eno3 | |   +----+------+---------+-------------------+-----------------------------------------+-----------+----------+----------+   | 3 | 0 | 03:00.3 | a0:d3:c1:01:43:af | NetXtreme BCM5719 Gigabit Ethernet PCIe | tg3 | eno4 | |   +----+------+---------+-------------------+-----------------------------------------+-----------+----------+----------+   | 4 | 0 | 07:00.0 | 7c:fe:90:20:e8:a0 | MT27700 Family [ConnectX-4] | mlx5_core | ens2f0 | |   +----+------+---------+-------------------+-----------------------------------------+-----------+----------+----------+   | 5 | 0 | 07:00.1 | 7c:fe:90:20:e8:a1 | MT27700 Family [ConnectX-4] | mlx5_core | ens2f1 | |   +----+------+---------+-------------------+-----------------------------------------+-----------+----------+----------+   Please choose even number of interfaces from the list above, either by ID , PCI or Linux IF   Stateful will use order of interfaces: Client1 Server1 Client2 Server2 etc. for flows.   Stateless can be in any order.   Enter list of interfaces separated by space (for example: 1 3) : 4 5   * Choosing 4 5 IDs for two ConnectX-4 ports to be used.       For interface 4, assuming loopback to it's dual interface 5.   Putting IP 1.1.1.1, default gw 2.2.2.2 Change it?(y/N).n   For interface 5, assuming loopback to it's dual interface 4.   Putting IP 2.2.2.2, default gw 1.1.1.1 Change it?(y/N).n   * y will allow you to edit the default IP values.           Print preview of generated config? (Y/n)y   ### Config file generated by dpdk_setup_ports.py ###           - port_limit: 2   version: 2   interfaces: ['07:00.0', '07:00.1']   port_info:   - ip: 1.1.1.1   default_gw: 2.2.2.2   - ip: 2.2.2.2   default_gw: 1.1.1.1           platform:   master_thread_id: 0   latency_thread_id: 10   dual_if:   - socket: 0   threads: [1,2,3,4,5,6,7,8,9,20,21,22,23,24,25,26,27,28]       Save the config to file? (Y/n)y   Save the config to file? (Y/n)y   Default filename is /etc/trex_cfg.yaml   Press ENTER to confirm or enter new file:

Run TRex server.

Count the number of threads automatically listed in previous step (2) marked in red (this example show 18 threads).

Enter the number of threads to the following command (-c <number_of_threads>) which not exceed 14 (if number of threads is above 14 enter 14).

CLI Command

Copy
Copied!
            

nohup ./t-rex-64 --no-ofed-check -i -c 14 &

In case the command above fails due unmatched CPU architecture you can run using this command.

CLI command

Copy
Copied!
            

nohup ./t-rex-64-o --no-ofed-check -i -c 14 &

Connecting GUI from windows desktop.

1. Download & install TRex GUI for windows to your desktop from here: https://github.com/cisco-system-traffic-generator/trex-stateless-gui/releases

2. Open the GUI and connect to T-rex server - File->Connect

image2019-7-11_14-1-5.png

3. Choose a port and press ’Acquire’

me.JPG

Amir Zeidner

For the past several years, Amir has worked as a Solutions Architect primarily in the Telco space, leading advanced solutions to answer 5G, NFV, and SDN networking infrastructures requirements. Amir’s expertise in data plane acceleration technologies, such as Accelerated Switching and Network Processing (ASAP²) and DPDK, together with a deep knowledge of open source cloud-based infrastructures, allows him to promote and deliver unique end-to-end NVIDIA Networking solutions throughout the Telco world.

Last updated on Sep 12, 2023.