NVIDIA DOCA DPA L2 Reflector Application Guide

1.0

This document provides a DPA L2 reflector implementation on top of the NVIDIA® BlueField®-3 DPU.

The BlueField-3 DPU supports high-speed Data Path Accelerator (DPA) . Data path accelerator allows for accelerated packet processing and manipulation.

DOCA l ayer-2 reflector uses the DPA engine to intercept network traffic and swap the source and destination MAC addresses of each packet.

The application accepts traffic from a specific port given as an argument and leverages DPA capabilities for accelerated processing.

The following figure provides a high-level view of the components of the application:

system-design-diagram-version-1-modificationdate-1702941436223-api-v2.png

DOCA L2 reflector runs on top of FlexIO SDK to configure the DPA engine.

architecture-diagram-version-1-modificationdate-1702941436773-api-v2.png

The FlexIO application consist of two parts:

  • Host side – responsible for allocating resources and loading them to the DPA

  • Device side – core processing logic of the application which swaps the MACs on the DPA

For more information, refer to "Programming FlexIO SDK".

This application leverages the following DOCA driver:

Refer to its programming guide for more information.

NVIDIA® BlueField®-3 DPU and above is required.

Installation

Please refer to the NVIDIA DOCA Installation Guide for Linux for details on how to install BlueField-related software.

Application Execution

The L2 reflector application is provided in both source and binary forms. The binary is located under /opt/mellanox/doca/applications/l2_reflector/bin/l2_reflector.

  1. Application usage instructions:

    Copy
    Copied!
                

    Usage: l2_reflector [DOCA Flags] [Program Flags]   DOCA Flags: -h, --help Print a help synopsis -v, --version Print program version information -l, --log-level Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE> --sdk-log-level Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE> -j, --json <path> Parse all command flags from an input json file   Program Flags: -d, --device <device name> Device name

    Note

    This usage printout can be printed to the command line using the -h (or --help) options:

    Copy
    Copied!
                

    /opt/mellanox/doca/applications/l2_reflector/bin/l2_reflector -h

    Note

    For additional information, refer to section "Command Line Flags".

  2. CLI example for running the application on BlueField or host:

    Copy
    Copied!
                

    /opt/mellanox/doca/applications/l2_reflector/bin/l2_reflector -d mlx5_0

    Warning

    The used device name (-d flag) must match the identifier of the desired IB device.

    Note

    To run the application on the second port, verify that it has a partition. Run:

    Copy
    Copied!
                

    dpaeumgmt partition info -d mlx5_1

    If DPA EU partition creation is required, refer to NVIDIA DOCA DPA Execution Unit Management Tool.

  3. The application also supports a JSON-based deployment mode, in which all command-line arguments are provided through a JSON file:

    Copy
    Copied!
                

    l2_reflector --json [json_file]

    For example:

    Copy
    Copied!
                

    cd /opt/mellanox/doca/applications/l2_reflector/bin ./l2_reflector --json ./l2_reflector_params.json

    Warning

    Before execution, ensure that the used JSON file contains the correct configuration parameters, and especially the desired PCIe addresses required for the deployment.

Command Line Flags

Flag Type

Short Flag

Long Flag/JSON Key

Description

JSON Content

General flags

h

help

Prints a help synopsis

N/A

v

version

Prints program version information

N/A

l

log-level

Set the log level for the application:

  • DISABLE=10

  • CRITICAL=20

  • ERROR=30

  • WARNING=40

  • INFO=50

  • DEBUG=60

  • TRACE=70 ( requires compilation with TRACE log level support )

Copy
Copied!
            

"log-level": 60

N/A

sdk-log-level

Sets the log level for the program:

  • DISABLE=10

  • CRITICAL=20

  • ERROR=30

  • WARNING=40

  • INFO=50

  • DEBUG=60

  • TRACE=70

Copy
Copied!
            

"sdk-log-level": 40

j

json

Parse all command flags from an input JSON file

N/A

Program flags

d

device

Device name

Copy
Copied!
            

"device": mlx5_0

Note

Refer to DOCA Arg Parser for more information regarding the supported flags and execution modes.


Troubleshooting

Warning

DPA L2 reflector works with packets with a specific source MAC address. To check the supported MAC address, refer to /opt/mellanox/doca/applications/l2_reflector/src/host/l2_reflector_core.h.

Please refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the installation or execution of the DOCA applications.

In addition to providing the application in binary form, the installation also includes all of the application sources and compilation instructions so as to allow modifying the sources and recompiling the application. For more information about the applications, as well as development and compilation tips, refer to the DOCA Applications page.

The sources of the application can be found under the /opt/mellanox/doca/applications/l2_reflector/src directory.

Recompiling All Applications

The applications are all defined under a single meson project, so the default compilation recompiles all the DOCA applications.

To build all the applications together, run:

Copy
Copied!
            

cd /opt/mellanox/doca/applications/ meson /tmp/build ninja -C /tmp/build

Note

l2_reflector is created under /tmp/build/l2_reflector/src/host.


Recompiling DPA L2 Reflector Application Only

To directly build only the L2 reflector application:

Copy
Copied!
            

cd /opt/mellanox/doca/applications/ meson /tmp/build -Denable_all_applications=false -Denable_l2_reflector=true ninja -C /tmp/build

Note

l2_reflector is created under /tmp/build/l2_reflector/src/host.

Alternatively, one can set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:

  1. Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:

    • Set enable_all_applications to false

    • Set enable_l2_reflector to true

  2. Run the following compilation commands :

    Copy
    Copied!
                

    cd /opt/mellanox/doca/applications/ meson /tmp/build ninja -C /tmp/build

    Note

    l2_reflector is created under /tmp/build/l2_reflector/src/host.

Troubleshooting

Refer to the NVIDIA DOCA Troubleshooting Guide for any issue you may encounter with the compilation of the DOCA applications.

This section lists the application's configuration flow which includes different FlexIO functions and wrappers.

  1. Parse application argument.

    1. Initialize arg parser resources and register DOCA general parameters.

      Copy
      Copied!
                  

      doca_argp_init();

    2. Register the application's parameters.

      Copy
      Copied!
                  

      register_l2_reflector_params();

    3. Parse the arguments.

      Copy
      Copied!
                  

      doca_argp_start();

  2. Setup the InfiniBand device.

    Copy
    Copied!
                

    l2_reflector_setup_ibv_device();

  3. Setup the DPA device.

    Copy
    Copied!
                

    l2_reflector_setup_device();

  4. Allocate the device's resources.

    Copy
    Copied!
                

    l2_reflector_allocate_device_resources();

  5. R un initialization function on the device.

    Copy
    Copied!
                

    flexio_process_call();

  6. Create the steering rule.

    Copy
    Copied!
                

    l2_reflector_create_steering_rule();

  7. Start the event handler on the device.

    Copy
    Copied!
                

    flexio_event_handler_run();

  8. Main loop.

    Copy
    Copied!
                

    while (!force_quit) sleep(10);

  9. Cleanup the resources.

    Copy
    Copied!
                

    l2_reflector_destroy();

  • /opt/mellanox/doca/applications/l2_reflector/src

  • /opt/mellanox/doca/applications/l2_reflector/bin/l2_reflector_params.json

© Copyright 2023, NVIDIA. Last updated on Feb 9, 2024.