What can I help you with?
DOCA Documentation v3.0.0
NVIDIA Docs Hub  NVIDIA Networking  BlueField DPUs / SuperNICs & DOCA  DOCA Documentation v3.0.0  NVIDIA DOCA IP Fragmentation Application Guide

On This Page

NVIDIA DOCA IP Fragmentation Application Guide

This guide provides an implementation of IP fragmentation application on top of NVIDIA® BlueField®-3 DPU.

Introduction

This IP Fragmentation application is designed to handle IP fragmentation and reassembly efficiently, ensuring minimal processing overhead for non-fragmented packets while maintaining high performance for fragmented packets.

The application operates on a multi-core architecture, uses Receive Side Scaling (RSS) to distribute traffic, and supports configurable modes for flexible port configurations.

Key Features:

  • IP Reassembly:

    • Functionality: The application assembles fragmented packets received on input ports based on their fragmentation headers.

    • Workflow: Upon successful reassembly, the complete packets are forwarded to their destination port.

  • IP Fragmentation:

    • Functionality: Packets exceeding a configurable Maximum Transmission Unit (MTU) are fragmented into smaller packets.

    • Workflow: Fragments are generated with correct headers and forwarded while maintaining efficient resource utilization.

  • Transparent Forwarding: Packets that are neither fragmented nor require reassembly are forwarded directly without additional processing overhead.

  • Inner and Outer Fragmentation Handling: The application supports handling fragmentation at both inner (e.g., encapsulated traffic like GRE, VXLAN) and outer IP layers.

  • Performance Optimization:

    • Designed for high throughput using multi-core processing.

    • Utilizes RSS to distribute traffic across multiple cores, ensuring efficient CPU utilization and scalability.

  • Debuggability with Counters.

  • Dual Operating Modes:

    • Dual Port Mode (Bidirectional): Traffic flows bidirectionally between two ports A ↔ B.

    • Quad Port Mode (Multiport): Independent unidirectional forwarding from Port A ↔ Port B and Port C ↔ Port D. In this mode, the direction of the traffic is isolated to go through two ports each time.

    • In both modes, non-fragmented or valid-sized packets follow the same path without additional actions.

System Design

The IP Fragmentation application can either run as a SW on BlueField-3's cores serving as an underlying service for host applications or on the host.

Application Architecture

Operational Workflow

  • Packet Reception and Classification:

    • Traffic is received on the input ports, with RSS distributing flows to the cores the applications runs on.

    • Packets are classified into three categories:

      • Fragmented (Needs Reassembly)

      • Too Large (Needs Fragmentation)

      • Standard Packets (Direct Forwarding)

  • Reassembly:

    • Fragments are buffered and reassembled using a configurable timeout.

    • Once reassembled, the full packet is validated and forwarded to the dedicated port (depending on the mode).

  • Fragmentation:

    • Large packets exceeding the MTU are fragmented.

    • Fragments are prepared with correct headers, sequence numbers, and size.

  • Direct Forwarding:

    • Standard packets are forwarded with minimal processing

Performance and Scalability

  • Multi-Core Processing: The application scales horizontally with the number of CPU cores, with each core handling a subset of traffic flows.

  • RSS Traffic Distribution: Receive Side Scaling ensures optimal load balancing across cores.

  • Minimal Overhead: Processing logic is optimized for low-latency handling of standard packets while ensuring efficient fragmentation and reassembly operations.

Debugging and Monitoring

Application provides real-time counters for insights for:

  • Packets processed.

  • Fragments reassembled or fragmented.

  • Errors such as timeout on incomplete fragments.

DOCA Libraries

This application leverages the following DOCA libraries:

For additional information about the used DOCA libraries, please refer to the respective programming guides.

Dependencies

  • NVIDIA BlueField-3 DPU is required.

  • Ubuntu 22.04 host (x86)

Compiling the Application

Please refer to the DOCA Installation Guide for Linux for details on how to install BlueField-related software.

The installation of DOCA's reference applications contains the sources of the applications, alongside the matching compilation instructions. This allows for compiling the applications "as-is" and provides the ability to modify the sources, then compile a new version of the application.

For more information about the applications as well as development and compilation tips, refer to the DOCA Reference Applications page.

The sources of the application can be found under the application's directory: /opt/mellanox/doca/applications/ip_frag/.

To build the IP fragmentation application:

Copy
Copied!
            

            
cd /opt/mellanox/doca/applications/
meson /tmp/build -Denable_all_applications=false -Denable_ip_frag=true
ninja -C /tmp/build

doca_ip_frag is created under /tmp/build/ip_frag/.

Alternatively, one can set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:

  1. Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:

    1. Set enable_all_applications to false

    2. Set enable_ip_frag to true

  2. Execute:

    Copy
    Copied!
                
    
            
    cd /opt/mellanox/doca/applications/
meson /tmp/build
ninja -C /tmp/build

    doca_ip_frag is created under /tmp/build/ip_frag/.

Troubleshooting

Please refer to the NVIDIA BlueField Platform Software Troubleshooting Guide for any issue you may encounter with the compilation of the DOCA applications.

Running the Application

Prerequisites

  • The Fragmentation application is based on DOCA Flow. Therefore, the user is required to allocate huge pages.

    Copy
    Copied!
                
    
            
    $ echo '4096' | sudo tee -a /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
$ sudo mkdir /mnt/huge
$ sudo mount -t hugetlbfs -o pagesize=2M nodev /mnt/huge

  • The Flex Parser Profile is a setting that enables flexible protocol parsing on NVIDIA NICs/DPUs. To enable GTP protocol support, set the Flex Parser Profile to 3. This configuration is mandatory and must be done manually in the system :

    Copy
    Copied!
                
    
            
    $ sudo mlxconfig -d <pcie_address> s FLEX_PARSER_PROFILE_ENABLE=3

Application Execution

The IP Fragmentation application is provided in source form, hence a compilation is required before the application can be executed.

  1. Application usage instructions:

    Copy
    Copied!
                
    
            
    Usage: doca_ip_frag [DPDK Flags] -- [DOCA Flags] [Program Flags]
 
DOCA Flags:
  -h, --help                        Print a help synopsis
  -v, --version                     Print program version information
  -l, --log-level                   Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
  --sdk-log-level                   Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
  -j, --json <path>                 Parse all command flags from an input json file
 
Program Flags:
  -m, --mode                        Ip_frag application mode. Bidirectional mode forwards packets between a single reassembly port and a single fragmentation port. Multiport mode forwards packets between two pairs of reassembly and fragmentation ports. For more information consult DOCA IP Fragmentation Application Guide. Format: bidir, multiport
  -u, --mtu                         MTU size
  -t, --frag-aging-timeout          Aging timeout of fragments pending packet reassembly in the fragmentation table (in ms)
  -s, --frag-tbl-size               Frag table size, i.e. maximum amount of concurrent defragmentation contexts per worker thread
  -c, --mbuf-chain                  Enable mbuf chaining

    For additional information, please refer to the Command Line Flags section below.

    The above usage printout can be printed to the command line using the -h (or --help) options:

    Copy
    Copied!
                
    
            
    /tmp/build/ip_frag/doca_ip_frag -- -h

  2. CLI example for running the application on BlueField-3:

    Copy
    Copied!
                
    
            
    /tmp/build/ip_frag/doca_ip_frag -a auxiliary:mlx5_core.sf.2,dv_flow_en=2,sft_en=1 -a auxiliary:mlx5_core.sf.4,dv_flow_en=2,sft_en=1  -a auxiliary:mlx5_core.sf.3,dv_flow_en=2,sft_en=1  -a auxiliary:mlx5_core.sf.5,dv_flow_en=2,sft_en=1  -l 3-15  --  -l 50 -m multiport

  3. CLI example for running the application on the host:

    Copy
    Copied!
                
    
            
    /tmp/build/ip_frag/doca_ip_frag -l 0-7 -a 0000:08:00.0,dv_flow_en=2 -a 0000:08:00.1,dv_flow_en=2 -- -l 60 -m bidir -t 1000

  4. The application also supports a JSON-based deployment mode, in which all command-line arguments are provided through a JSON file:

    Copy
    Copied!
                
    
            
    /tmp/build/ip_frag/doca_ip_frag --json [json_file]

    For example:

    Copy
    Copied!
                
    
            
    /tmp/build/ip_frag/doca_ip_frag --json /opt/src/doca/applications/ip_frag/ip_frag_params.json

    Before execution, please ensure that the used JSON file contains the correct configuration parameters, and especially the desired PCI addresses needed for the deployment.

Command Line Flags

Flag Type

Short Flag

Long Flag/JSON Key

Description

JSON Content

DPDK flags

a

devices

Add devices to the allow list

This is a mandatory flag.
Copy
Copied!
            

            
"devices": [
    {
        "device": "pf",
        "id": "0000:08:00.0",
        "hws": true,
    },
    {
        "device": "pf",
        "id": "0000:08:00.1",
        "hws": true,
    },
],

l

core-list

List of cores to be used by the application data path

This is a mandatory flag.
Copy
Copied!
            

            
"core-list": "0-1"

General flags

h

help

Prints a help synopsis

N/A

v

version

Prints program version information

N/A

l

log-level

Set the log level for the application:

  • DISABLE=10

  • CRITICAL=20

  • ERROR=30

  • WARNING=40

  • INFO=50

  • DEBUG=60

  • TRACE=70 (Requires compilation with Trace level support)
Copy
Copied!
            

            
"log-level": 60

N/A

sdk-log-level

Sets the log level for the program:

  • DISABLE=10

  • CRITICAL=20

  • ERROR=30

  • WARNING=40

  • INFO=50

  • DEBUG=60

  • TRACE=70
Copy
Copied!
            

            
"sdk-log-level": 40

j

json

Parse all command flags from an input json file

N/A

Program flags

m

mode

Execution mode: bidir, multiport

This is a mandatory flag.
Copy
Copied!
            

            
"mode": "bidir"

u

mtu

MTU for fragmentation
Copy
Copied!
            

            
"mtu": 1518

t

frag-aging-timeout

Fragmentation table aging timeout (in milliseconds)
Copy
Copied!
            

            
"frag-aging-timeout": 2

s

frag-tbl-size

Fragmentation table size
Copy
Copied!
            

            
"frag-tbl-size": 2048

c

mbuf-chain

Enable mbuf chaining support on packet reassembly
Copy
Copied!
            

            
"mbuf-chain": false

Refer to DOCA Arg Parser for more information regarding the supported flags and execution modes.

Troubleshooting

Please refer to the NVIDIA BlueField Platform Software Troubleshooting Guide for any issue you may encounter with the installation or execution of the DOCA applications.

References

  • /opt/mellanox/doca/applications/ip_frag/

  • /opt/mellanox/doca/applications/ip_frag/ip_frag_params.json
© Copyright 2025, NVIDIA. Last updated on May 5, 2025.
content here