DOCA Eth L2 Forwarding Application Guide

This document provides an Ethernet L2 Forwarding implementation on top of the NVIDIA® BlueField® DPU.

Th e Ethernet L2 Forwarding applicatio n is a DOCA Ethernet based application that forwards traffic from a single RX port to a single TX port and vice versa, leveraging DOCA's task/event batching feature for enhanced performance.

The application can run both on the host and the BlueField, and has two main modes:

• Two-sided forwarding – device 1 → device 2 and device 2 → device 1

• One-sided forwarding – device 1 → device 2 or device 2 → device 1

The one-sided mode offers better performance, enlarging the packets forwarding rate.

The Ethernet L2 Forwarding application runs on the host or the BlueField.

• The following diagram shows the application running on the host.

  

• The following diagram shows the application running on the BlueField:

  

The Ethernet L2 Forwarding application runs on top of the DOCA Ethernet API to form an (two/one-sided) L2 forwarding between two ports.

1. Two DOCA devices are opened.

2. Two DOCA mmaps are created.

3. Two DOCA Flow ports are configured and started, each with a different opened DOCA device.

4. Two DOCA Ethernet TXQ and RXQ contexts are initialized, each TXQ-RXQ pair with a different opened DOCA device such that traffic is steered from the device to the corresponding RXQ, and from the corresponding TXQ to the device.

5. Forwarding - Packets received by device x are steered to RXQ x, then allocated to TXQ y and sent by device y (and vice versa).

This application leverages the following DOCA libraries:

• DOCA Ethernet

• DOCA Flow

For additional information about the used DOCA libraries, please refer to the respective programming guides.

DOCA reference applications are installed with full source code and build instructions. This allows you to compile them as-is or modify the source code to create custom versions.

The source code for the application is located in the following directory:

            

            
/opt/mellanox/doca/applications/eth_l2_fwd/
        

Compiling All Applications

The applications are all defined under a single meson project, meaning that the default compilation will compile all the DOCA applications.

To build all the applications together, run:

            

            
cd /opt/mellanox/doca/applications/
meson /tmp/build 
ninja -C /tmp/build
        

Compiling Only the Current Application

1. To directly build only the Ethernet L2 Forwarding application:

   Copy

   Copied!

               
   
               
   cd /opt/mellanox/doca/applications/
   meson /tmp/build -Denable_all_applications=false -Denable_eth_l2_fwd=true
   ninja -C /tmp/build
           

   Info

   doca_eth_l2_fwd will be created under /tmp/build/eth_l2_fwd/.

2. Alternatively, one can set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:

   1. Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:

      • Set enable_all_applications to false

      • Set enable_eth_l2_fwd to true

   2. The same compilation commands should be used, as were shown in the previous section:

      Copy

      Copied!

                  
      
                  
      cd /opt/mellanox/doca/applications/
      meson /tmp/build 
      ninja -C /tmp/build
              

      Info

      doca_eth_l2_fwd will be created under /tmp/build/eth_l2_fwd/.

Application Execution

The Ethernet L2 Forwarding application is provided in source form, hence a compilation is required before the application can be executed.

• Application usage instructions:

  Copy

  Copied!

              
  
              
  Usage: doca_eth_l2_fwd [DOCA Flags] [Program Flags]
   
  DOCA Flags:
    -h, --help                        Print a help synopsis
    -v, --version                     Print program version information
    -l, --log-level                   Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
    --sdk-log-level                   Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
    --log-filter                      Filter logs from specific modules, separated by comma
    -j, --json <path>                 Parse command line flags from an input json file
   
  Program Flags:
    -d, --devs-names <name1,name2>    Set two IB devices names separated by a comma, without spaces.
    -r, --rate <rate>                 Set packets receive rate (in [MB/s]), default is 12500.
    -ps, --pkt-size <size>            Set max packet size (in [B]), default is 1600.
    -t, --time <time>                 Set packet max process time (in [μs]), default is 1.
    -nt, --num-tasks <num>            Set number of tasks per batch, default is 128.
    -nb, --num-batches <num>          Set number of task batches, default is 32.
    -o, --one-sided-forwarding <num>  Set one-sided forwarding: 0 - two-sided forwarding, 1 - device 1 -> device 2, 2 - device 2 -> device 1. default is 0.
    -f, --max-forwardings <num>       Set max forwardings after which the application run will end, default is 0, meaning no limit.
          

  For additional information, please refer to the Command Line Flags section below.

  Note

  The above usage printout can be printed to the command line using the -h (or --help) options:

  Copy

  Copied!

              
  
              
  ./doca_eth_l2_fwd -h
          

• CLI example for running the application either on the BlueField or on the host:

  Copy

  Copied!

              
  
              
  ./doca_eth_l2_fwd -d mlx5_0,mlx5_1
          

  Note

  Both IB devices identifiers ( mlx5_0, mlx5_1 ) should match the identifiers of the desired IB devices.

Command Line Flags

General Flags

Program Flags

Troubleshooting

Refer to the NVIDIA BlueField Platform Software Troubleshooting Guide for any issue encountered with the compilation, installation, or execution of the DOCA applications.

1. Parse application argument.

   1. Initialize Arg parser resources and register DOCA general parameters.

      Copy

      Copied!

                  
      
                  
      doca_argp_init();
              

   2. Register Ethernet L2 Forwarding application parameters.

      Copy

      Copied!

                  
      
                  
      register_eth_l2_forwarding_params();
              

   3. Parse the arguments.

      Copy

      Copied!

                  
      
                  
      doca_argp_start();
              

      1. Parse DOCA flags.

      2. Parse application parameters.

2. Execute Ethernet L2 Forwarding application main logic.

   Copy

   Copied!

               
   
               
   eth_l2_fwd_execute();
           

   1. Open the two chosen DOCA devices.

   2. Initialize necessary DOCA Core objects.

   3. Initialize ETH RXQ/TXQ contexts for the devices.

   4. Forward packets.

3. Clean up application resources.

   Copy

   Copied!

               
   
               
   eth_l2_fwd_cleanup();
           

   1. Stop all contexts and drain tasks.

   2. Free all application resources.

4. Arg parser destroy.

   Copy

   Copied!

               
   
               
   doca_argp_destroy()
           

• /opt/mellanox/doca/applications/eth_l2_fwd/