What can I help you with?
DOCA Documentation v3.0.0

DOCA Flow Connection Tracking

This guide provides an overview and configuration instructions for the DOCA Flow Connection Tracking (CT) API.

The DOCA Flow Connection Tracking (CT) module is a 5-tuple table designed to efficiently track network connections using hardware resources. It supports the following key features:

  • Track zone and 5-tuple sessions – Track and manage network connections based on a 5-tuple (source IP, destination IP, source port, destination port, and protocol) along with zone-based separation

  • Zone-based virtual tables – Enable logical isolation using zones

  • Aging support – Remove idle connections automatically using configurable timeouts

  • Connection metadata – Set and manage metadata for tracked connections

  • Bidirectional packet handling – Manage traffic in both directions of a connection

  • High connection rate – Efficiently handle a high rate of connections per second (CPS)

The CT module makes it simple and efficient to track connections by leveraging hardware resources.

The DOCA Flow CT Pipe is designed to handle non-encapsulated TCP and UDP packets. It supports two primary actions:

  • Forward to next pipe – For packets that match a known 6-tuple connection (5-tuple + zone)

  • Miss to next pipe – For packets without a matching connection entry

The application is responsible for handling packets based on these outcomes.

The DOCA Flow CT API consists of four major components:

  • CT module manipulation – Configure and manage resources within the CT module

  • CT connection entry manipulation – Add, remove, or update connection entries efficiently

  • Callbacks – Handle asynchronous processing results for connection entries

  • Pipe and entry statistics – Monitor connection tracking performance using pipe-level and entry-level statistics

These components provide flexible control over connection tracking and monitoring, allowing applications to adapt to various network scenarios effectively.

arch-diagram-version-1-modificationdate-1739717862073-api-v2.png

Aging

Aging time refers to the maximum duration (in seconds) a session can remain active without detecting any packets. If no packets are observed within this period, the session is terminated.

To support aging, a dedicated aging thread is launched. This thread polls and checks counters for all active connections, ensuring that stale sessions are removed efficiently.

Note

When aging is enabled, either the counter flag or a non-zero timeout must be set for at least one connection entry to trigger session expiration.


Managed Mode

In Managed Mode, the application is responsible for:

  • Managing worker threads

  • Parsing and handling connection lifecycles

This mode utilizes DOCA Flow CT management APIs for creating and destroying connections.

The CT aging module automatically notifies the application of aged-out connections by invoking callbacks.

Connection Rules and Management

Users have the flexibility to create connection rules with different patterns, metadata (meta), or counters which can be applied separately for each packet direction.

Note

Users must manually define the appropriate meta and mask values for matching ( match ) and modifying ( modify ) packets.

To create rules in stages:

  1. Create one rule for a connection using the standard API.

  2. Add a second rule for the opposite packet direction using the doca_flow_ct_entry_add_dir() API.

managed-mode-diagram-version-1-modificationdate-1739717861293-api-v2.png

Processing CT Entries

DOCA Flow provides specialized APIs to process CT entries using a dedicated queue:

  • doca_flow_entries_process – Processes pipe entries in the queue

  • doca_flow_aging_handle – Handles the aging of pipe entries

Info

Some APIs, such as CT entry status queries and pipe miss queries, are not supported in Managed Mode.


DPU

To enable DOCA Flow CT on the DPU, perform the following on the Arm:

  1. Enable iommu.passthrough in Linux boot commands ( or disable SMMU from the DPU BIOS):

    1. Run:

      Copy
      Copied!
                  

      sudo vim /etc/default/grub

    2. Set GRUB_CMDLINE_LINUX="iommu.passthrough=1".

    3. Run:

      Copy
      Copied!
                  

      sudo update-grub sudo reboot

  2. Configure DPU firmware with LAG_RESOURCE_ALLOCATION=1:

    Copy
    Copied!
                

    sudo mlxconfig -d <device-id> s LAG_RESOURCE_ALLOCATION=1

    Info

    Retrieve device-id from the output of the mst status -v command. If, under the MST tab, the value is N/A, run the mst start command.

  3. Update /etc/mellanox/mlnx-bf.conf as follows:

    Copy
    Copied!
                

    ALLOW_SHARED_RQ="no"

  4. Perform power cycle on the host and Arm sides.

  5. If working with a single port, set the DPU into e-switch mode:

    Copy
    Copied!
                

    sudo devlink dev eswitch set pci/<pcie-address> mode switchdev sudo devlink dev param set pci/<pcie-address> name esw_multiport value false cmode runtime

    Info

    Retrieve pcie-address from the output of the mst status -v command.

  6. If working with two PF ports, set the DPU into multi-port e-switch mode (for the 2 PCIe devices):

    Copy
    Copied!
                

    sudo devlink dev param set pci/<pcie-address> name esw_multiport value true cmode runtime

    Info

    Retrieve pcie-address from the output of the mst status -v command.

  7. Define huge pages (see DOCA Flow prerequisites).

ConnectX

To enable DOCA Flow CT on the NVIDIA® ConnectX®, perform the following:

  1. Configure firmware with LAG_RESOURCE_ALLOCATION=1:

    Copy
    Copied!
                

    sudo mlxconfig -d <device-id> s LAG_RESOURCE_ALLOCATION=1

    Info

    Retrieve device-id from the output of the mst status -v command. If, under the MST tab, the value is N/A, run the mst start command.

  2. Perform power cycle.

  3. If working with a single port:

    Copy
    Copied!
                

    sudo devlink dev eswitch set pci/<pcie-address> mode switchdev sudo devlink dev param set pci/<pcie-address> name esw_multiport value false cmode runtime

    Info

    Retrieve pcie-address from the output of the mst status -v command.

  4. If working with two PF ports:

    Copy
    Copied!
                

    sudo devlink dev eswitch set pci/<pcie-address0> mode switchdev sudo devlink dev eswitch set pci/<pcie-address1> mode switchdev sudo devlink dev param set pci/<pcie-address0> name esw_multiport value true cmode runtime sudo devlink dev param set pci/<pcie-address1> name esw_multiport value true cmode runtime

    Info

    Retrieve pcie-address from the output of the mst status -v command.

  5. Define huge pages (see DOCA Flow prerequisites).

DOCA Flow CT supports actions based on meta and NAT operations. Each action can be defined as either shared or non-shared.

Note

Action descriptors are not supported.

Shared Actions

Actions that can be shared between entries. Shared actions are predefined and reused in multiple entries.

The user gets a handle per shared action created and uses this handle as a reference to the action where required.

Info

It is user responsibility to track shared actions and to remove them when they become irrelevant.

Shared actions are defined using a control queue (see struct doca_flow_ct_cfg).

Non-shared Actions

Actions provided with their data during entry create/update.

These actions are completely managed by DOCA Flow CT and cannot be reused in multiple flows (i.e., NAT operations).

Action Sets in Pipe Creation

When creating a DOCA Flow CT pipe, users must define action sets, just as they would for any other pipe.

Fields in the CT pipe must be marked as CHANGEABLE during pipe creation. This allows the actual criteria for these fields to be specified later during entry creation.

Info

Only actions related to meta and NAT, as defined in struct doca_flow_ct_actions , are supported.

During entry creation or update, different actions can be specified for each direction, allowing variations in action content and/or action type.

Feature Enable

To enable user actions, configure the following parameters:

  • User action templates during DOCA Flow CT pipe creation

  • Maximum number of user actions (nb_user_actions on DOCA Flow CT init)

Using Actions in Managed Mode

Init

Configure the following parameters on doca_flow_ct_init():

  • nb_ctrl_queues – number of control queues for defining shared actions

  • nb_user_actions – maximum number of user actions. Must align to 64. Both shared control queues and non-shared control queues cache action IDs to speed up ID allocation. Each queue may cache a maximum of 1024 IDs. Users must configure the expected number of actions + total queues * 1024. This number cannot exceed the number of actions hardware supports.

Create DOCA Flow CT Pipe

Configure actions sets on doca_flow_pipe_create().

Create Shared Actions

Use doca_flow_ct_actions_add_shared() with one of the control queues.

Shared actions can be added at any time before use.

Add Entry

Entry can be created in one of the following ways:

  • Using an action handle of a predefined shared action

  • Using action data, which is specific to the flow, not sharable (e.g., for NAT operations)

The entry can have different actions and/or different action types per direction.

Remove Entry

Non-shared actions associated with an entry are implicitly destroyed by DOCA Flow CT.

Shared actions are not destroyed. They can be used by the user until they decide to remove them.

Update Entry

Entry actions can be updated per direction. All combinations of shared/non-shared actions are applicable (e.g., update from shared to non-shared).

DOCA Flow CT allows using a different forward pipe per flow direction.

DOCA Flow CT supports the forward pipe in two levels:

  • Pipe level – a single forward pipe defined during DOCA Flow CT pipe creation and used for all entries

  • Entry level – forward pipe defined during entry create

  • DOCA Flow CT operates in one of the two levels

DOCA CT forward in entry level has the following characteristics:

  • Supports only DOCA_FLOW_FWD_PIPE (up to 4 different forward pipes)

  • Supports forward pipe per flow direction (both directions can have same/different forward pipe)

  • Must set forward pipes on each entry create (no default forward pipe)

Turn on the feature:

  1. Create DOCA Flow CT pipe with forward type = DOCA_FLOW_FWD_PIPE and next_pipe = NULL .

  2. Call to doca_flow_ct_fwd_register to register forward pipes and get fwd_handles in return.

Using Changeable Forward in Managed Mode

  1. Initialize DOCA Flow CT (doca_flow_ct_init​).

  2. Register forward pipes (doca_flow_ct_fwd_register​).

    • Define pipes that can be used for forward​

  3. Create DOCA Flow CT pipe​ (doca_flow_pipe_create) w ith definition of possible forward pipes.

  4. Add entry (doca_flow_ct_add_entry).

    • Set origin and/or reply fwd_handles returned from doca_flow_ct_fwd_register.

  5. Update forward for entry direction (doca_flow_ct_update_entry​).

    Note

    Updating forward handle requires setting all other parameters with their previous values.​

For the library API reference, refer to DOCA Flow and CT API documentation in the DOCA Library APIs.

Note

The pkg-config (*.pc file) for the Flow CT library is included in DOCA's regular definitions :doca.

The following sections provide additional details about the library API.

enum doca_flow_ct_flags

DOCA Flow CT configuration optional flags.

Flag

Description

DOCA_FLOW_CT_FLAG_STATS = 1u << 0

Enable internal pipe counters for packet tracking purposes. Call doca_flow_pipe_dump(<ct_pipe>) to dump counter values. Each call dumps values changed.

DOCA_FLOW_CT_FLAG_WORKER_STATS = 1u << 1,

Enable worker thread internal debug counter periodical dump.

DOCA_FLOW_CT_FLAG_NO_AGING = 1u << 2,

Disable aging

DOCA_FLOW_CT_FLAG_ASYMMETRIC_TUNNEL = 1u << 3,

Allow Tunnel or non-tunnel in different direction.

DOCA_FLOW_CT_FLAG_NO_COUNTER = 1u << 4,

Disable counter and aging to save aging thread CPU cycles

DOCA_FLOW_CT_FLAG_CT_PIPE_ONLY = 1u << 5,

The user must check TCP SYN flags and UDP in CT miss flow to identify ADD type packets.

DOCA_FLOW_CT_FLAG_WIRE_TO_WIRE = 1u << 6,

Hint traffic comes from uplink wire and forwards to uplink wire.

Note

If this flag is set, the direction info must be DOCA_FLOW_DIRECTION_NETWORK_TO_HOST.

DOCA_FLOW_CT_FLAG_DUP_FILTER_UDP_ONLY= 1u << 7,

Apply the connection duplication filter for UDP connections only


enum doca_flow_ct doca_flow_ct_entry_flags

DOCA Flow CT Entry optional flags.

Flag

Description

DOCA_FLOW_CT_ENTRY_FLAGS_NO_WAIT = (1 << 0)

Entry is not buffered; send to hardware immediately

DOCA_FLOW_CT_ENTRY_FLAGS_DIR_ORIGIN = (1 << 1)

Apply flags to origin direction

DOCA_FLOW_CT_ENTRY_FLAGS_DIR_REPLY = (1 << 2)

Apply flags to reply direction

DOCA_FLOW_CT_ENTRY_FLAGS_IPV6_ORIGIN = (1 << 3)

Origin direction is IPv6; origin match union in struct doca_flow_ct_match is IPv6

DOCA_FLOW_CT_ENTRY_FLAGS_IPV6_REPLY = (1 << 4)

Reply direction is IPv6; reply match union in struct doca_flow_ct_match is IPv6

DOCA_FLOW_CT_ENTRY_FLAGS_COUNTER_ORIGIN = (1 << 5)

Apply counter to origin direction

DOCA_FLOW_CT_ENTRY_FLAGS_COUNTER_REPLY = (1 << 6)

Apply counter to reply direction

DOCA_FLOW_CT_ENTRY_FLAGS_COUNTER_SHARED = (1 << 7)

Counter is shared for both direction (origin and reply)

DOCA_FLOW_CT_ENTRY_FLAGS_FLOW_LOG = (1 << 8)

Enable flow log on entry removed

DOCA_FLOW_CT_ENTRY_FLAGS_ALLOC_ON_MISS = (1 << 9)

Allocate on entry not found when calling doca_flow_ct_entry_prepare() API

DOCA_FLOW_CT_ENTRY_FLAGS_DUP_FILTER_ORIGIN = (1 << 10)

Enable duplication filter on origin direction

DOCA_FLOW_CT_ENTRY_FLAGS_DUP_FILTER_REPLY = (1 << 11)

Enable duplication filter on reply direction


enum doca_flow_ct_rule_opr

Options for handling flows in autonomous mode with shared actions. The decision is taken on the first flow packet.

Operation

Description

DOCA_FLOW_CT_RULE_OK

Flow should be defined in the CT pipe using the required shared actions handles

DOCA_FLOW_CT_RULE_DROP

Flow should not be defined in the CT pipe. The packet should be dropped.

DOCA_FLOW_CT_RULE_TX_ONLY

Flow should not be defined in the CT pipe. The packet should be transmitted.


struct direction_cfg

Managed mode configuration for origin or reply direction.

Field

Description

bool match_inner

5-tuple match pattern applies to packet inner layer

struct doca_flow_meta *zone_match_mask

Mask to indicate meta field and bits to match

struct doca_flow_meta *meta_modify_mask

Mask to indicate meta field and bits to modify on connection packet match


doca_flow_ct_cfg

DOCA Flow CT configuration manipulation:

Copy
Copied!
            

struct doca_flow_ct_cfg *ct_cfg;   ret = doca_flow_ct_cfg_create(&ct_cfg);   doca_flow_ct_cfg_set_flags(ct_cfg, flags); doca_flow_ct_cfg_set_queues(ct_cfg, n_queues); doca_flow_ct_cfg_set_connections(ct_cfg, n_v4, n_v6, n_max); ...   ret = doca_flow_ct_init(ct_cfg);   final: ret = doca_flow_ct_cfg_destroy(ct_cfg);

Where:

Field

Description

doca_flow_ct_cfg_create(struct doca_flow_ct_cfg **cfg)

Create CT configuration

doca_flow_ct_cfg_destroy(struct doca_flow_ct_cfg *cfg)

Destroy CT configuration

doca_flow_ct_cfg_set_flags(struct doca_flow_ct_cfg *cfg, uint32_t flags)

Set CT flags; see enum doca_flow_ct_flags

doca_flow_ct_cfg_set_queues(struct doca_flow_ct_cfg *cfg, uint32_t n_queues)

Set number of hardware queues to manipulate connections

doca_flow_ct_cfg_set_queue_depth(struct doca_flow_ct_cfg *cfg, uint32_t queue_depth);

Set queue depth; default to 512 rules

doca_flow_ct_cfg_set_ctrl_queues(struct doca_flow_ct_cfg *cfg, uint32_t n_ctrl_queues)

Number of CT control queues used for defining shared actions

doca_flow_ct_cfg_set_user_actions(struct doca_flow_ct_cfg *cfg, uint32_t n_user_actions)

Maximum number of user actions supported (shared and non-shared).

Minimum value is 1K * (nb_ctrl_queues + nb_arm_queues ).

doca_flow_ct_cfg_set_connections(struct doca_flow_ct_cfg *cfg,

uint32_t n_ipv4,

uint32_t n_ipv6,

uint32_t n_total)

Maximum number of IPv4, IPv6, and total CT connections

doca_flow_ct_cfg_set_max_connections_per_zone(struct doca_flow_ct_cfg *cfg,

uint32_t max_connections_per_zone)

Max CT connection per zone

doca_flow_ct_cfg_set_counter_asymmetric(struct doca_flow_ct_cfg *cfg, uint32_t n_counter_asymmetric)

Max connections with an asymmetric counter; non-shared counter

doca_flow_ct_cfg_set_entry_private_data_size(struct doca_flow_ct_cfg *cfg, uint32_t entry_priv_data_size)

User private data size to allocate per connection

doca_flow_ct_cfg_set_entry_finalize_cb(struct doca_flow_ct_cfg *cfg,

doca_flow_ct_entry_finalize_cb entry_finalize_cb)

Entry finalize callback to query connection final stats

doca_flow_ct_cfg_set_status_update_cb(struct doca_flow_ct_cfg *cfg,

doca_flow_ct_stats_update_cb stats_update_cb);

Status update callback to notify on counter changes

doca_flow_ct_cfg_set_aging_core(struct doca_flow_ct_cfg *cfg, uint32_t aging_core)

CPU core ID for CT aging thread to bind.

doca_flow_ct_cfg_set_aging_query_delay(struct doca_flow_ct_cfg *cfg, uint32_t aging_query_delay_s)

CT aging query delay for new created connections

doca_flow_ct_cfg_set_aging_plugin_ops(struct doca_flow_ct_cfg *cfg,

struct doca_flow_ct_aging_ops *aging_ops)

User-defined aging logic callback functions. Fallback to default aging logic

doca_flow_ct_cfg_set_dup_filter_size(struct doca_flow_ct_cfg *cfg, uint32_t dup_filter_sz)

Number of connections to cache in the duplication filter

doca_flow_ct_cfg_set_direction(struct doca_flow_ct_cfg *cfg,

bool direction,

bool match_inner,

struct doca_flow_meta *zone_match_mask,

struct doca_flow_meta *meta_modify_mask

Configuration origin and reply direction


struct doca_flow_ct_actions

This structure is used in the following cases:

  • For defining shared actions. In this case, action data is provided by the user. The action handle is returned by DOCA Flow CT.

  • For defining an entry with actions. The structure can be filled with two options:

    • With action handle of a previously created shared action

    • With non-shared action data

DOCA Flow CT action structure.

Copy
Copied!
            

enum doca_flow_resource_type  resource_type; union { /* Used when creating an entry with a shared action. */               uint32_t action_handle;   /* Used when creating an entry with non-shared action or when creating a shared action. */              struct {                           uint32_t action_idx;                         struct doca_flow_meta meta;                         struct doca_flow_header_l4_port l4_port;                         union {                                   struct doca_flow_ct_ip4 ip4;                                   struct doca_flow_ct_ip6 ip6;                          };              } data;      };

Where:

Field

Description

enum doca_flow_resource_type resource_type

Shared/non-shared action

uint32_t action_handle

Shared action handle

uint32_t action_idx

Actions template index

struct doca_flow_meta meta

Modify meta values

struct doca_flow_header_l4_port l4_port

UDP or TCP source and destination port

struct doca_flow_ct_ip4 ip4

Source and destination IPv4 addresses

struct doca_flow_ct_ip6 ip6

Source and destination IPv6 addresses

Info

The value in meta , l4_port , ip4 , and ip6 should start from bit0, the least significant bit, regardless of which bits are set in mask. For example, action_val.meta.u32[0] = DOCA_HTOBE32(0x12), action_mask.meta.u32[0] = DOCA_HTOBE32(0x0000FF00) sets bits 15-8 to 0x12.

This section describes DOCA Flow CT samples based on the DOCA Flow CT pipe.

The samples illustrate how to use the library API to manage TCP/UDP connections.

Info

All the DOCA samples described in this section are governed under the BSD-3 software license agreement.

Running the Samples

  1. Refer to the following documents:

  2. To build a given sample, run the following command. If you downloaded the sample from GitHub, update the path in the first line to reflect the location of the sample file:

    cd /opt/mellanox/doca/samples/doca_flow/flow_ct_udp

    meson /tmp/build

    ninja -C /tmp/build

    Info

    The binary doca_flow_ct_udp is created under /tmp/build/samples/.

  3. Sample (e.g., doca_flow_ct_udp ) usage:

    Usage: doca_<sample_name> [DOCA Flags] [Program Flags]

    DOCA Flags:

    -h, --help Print a help synopsis

    -v, --version Print program version information

    -l, --log-level Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>

    --sdk-log-level Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>

    -j, --json <path> Parse all command flags from an input json file

    Program Flags:

    -p, --pci_addr <PCI-ADDRESS> PCI device address

  4. For additional information per sample, use the -h option:

    /tmp/build/samples/<sample_name> -h

The following is a CLI example for running the samples when port 03:00.0 is configured (multi-port e-switch) as manager port:

/tmp/build/samples/doca_<sample_name> -- -p 03:00.0 -l 60

Info

To avoid the test being impacted by unexpected packets, it only accepts packets like the following examples:

  • IPv4 destination address is 1.1.1.1

  • IPv6 destination address is 0101:0101:0101:0101:0101:0101:0101:0101


Samples

Note
  • All CT UDP samples demonstrate the usage of the connection's duplication filter. Duplication filter is used if the user is interested in preventing same connection rule insertion in a high-rate workload environment.

  • These samples are also available on GitHub.

Flow CT 2 Ports

This sample illustrates how to create a simple pipeline on two standalone e-switches. M ulti-port e-switch must be disabled.

sudo devlink dev eswitch set pci/<pcie-address0> mode switchdev

sudo devlink dev eswitch set pci/<pcie-address1> mode switchdev

sudo devlink dev param set pci/<pcie-address0> name esw_multiport value false cmode runtime

The sample logic includes:

  1. Initializing DOCA Flow by indicating mode_args="switch,hws" in the doca_flow_cfg struct.

  2. Initializing DOCA Flow CT.

  3. Starting two DOCA Flow uplink ports where port 0 and 1 each has a special role of being a switch manager port.

    Info

    Ports are configured according to the parameters provided to doca_dpdk_port_probe() in the main function.

  4. Creating a pipeline on each port:

    1. Building an UDP pipe to filter non-UDP packets.

    2. Building a CT pipe to hold UDP session entries.

    3. Building a counter pipe with an example 5-tuple entry to which non-unidentified UDP sessions should be sent.

    4. Building a hairpin pipe to send back packets.

    5. Building an RSS pipe from which all packets are directed to the sample main thread for parsing and processing.

  5. Packet processing on each port:

    1. The first UDP packet triggers the miss flow as the CT pipe is empty.

    2. Performing 5-tuple packet parsing.

    3. Calling doca_flow_ct_add_entry() to create a hardware rule according to the parsed 5-tuple info.

    4. The second UDP packet based on the the same 5-tuple should be sent again. Packet hits the hardware rule inserted before and sent back to egress.

Reference:

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/flow_ct_2_ports_sample.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/flow_ct_2_ports_main.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/meson.buildFlow CT UDP

This sample illustrates how to create a simple UDP pipeline with a CT pipe in it.

The sample logic includes:

  1. Initializing DOCA Flow by indicating mode_args="switch,hws" in the doca_flow_cfg struct.

  2. Initializing DOCA Flow CT .

  3. Starting two DOCA Flow uplink representor ports where port 0 has a special role of being a switch manager port.

    Ports are configured according to the parameters provided to doca_dpdk_port_probe() in the main function.

  4. Creating a pipeline on the main port:

    1. Building an UDP pipe to filter non-UDP packets.

    2. Building a CT pipe to hold UDP session entries.

    3. Building a counter pipe with an example 5-tuple entry to which non-unidentified UDP sessions should be sent.

    4. Building a VXLAN encapsulation pipe to encapsulate all identified UDP sessions.

    5. Building an RSS pipe from which all packets are directed to the sample main thread for parsing and processing.

  5. Packet processing:

    1. The first UDP packet triggers the miss flow as the CT pipe is empty.

    2. 5-tuple packet parsing is performed.

    3. doca_flow_ct_add_entry() is called to create a hardware rule according to the parsed 5-tuple info.

    4. The second UDP packet based on the the same 5-tuple should be sent again. Packet hits the HW rule inserted before and directed to port 0 after VXLAN encapsulation.

Reference:

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/flow_ct_udp_sample.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/flow_ct_udp_main.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/meson.build

Flow CT UDP

This sample illustrates how to create a simple UDP pipeline with a CT pipe in it.

The sample logic includes:

  1. Initializing DOCA Flow by indicating mode_args="switch,hws" in the doca_flow_cfg struct.

  2. Initializing DOCA Flow CT .

  3. Starting two DOCA Flow uplink representor ports where port 0 has a special role of being a switch manager port.

    Info

    Ports are configured according to the parameters provided to doca_dpdk_port_probe() in the main function.

  4. Creating a pipeline on the main port:

    1. Building an UDP pipe to filter non-UDP packets.

    2. Building a CT pipe to hold UDP session entries.

    3. Building a counter pipe with an example 5-tuple entry to which non-unidentified UDP sessions should be sent.

    4. Building a VXLAN encapsulation pipe to encapsulate all identified UDP sessions.

    5. Building an RSS pipe from which all packets are directed to the sample main thread for parsing and processing.

  5. Packet processing:

    1. The first UDP packet triggers the miss flow as the CT pipe is empty.

    2. 5-tuple packet parsing is performed.

    3. doca_flow_ct_add_entry() is called to create a hardware rule according to the parsed 5-tuple info.

    4. The second UDP packet based on the the same 5-tuple should be sent again. Packet hits the HW rule inserted before and directed to port 0 after VXLAN encapsulation.

Reference:

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/flow_ct_udp_sample.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/flow_ct_udp_main.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp/meson.build

Flow CT UDP Query

This sample illustrates how to query a Flow CT UDP session entry. The query can be done according to session direction (origin or reply). The pipeline is identical to that of the Flow CT UDP sample.

This sample adds the following logic:

  1. Dumping port 0 information into a file at ./port_0_info.txt.

  2. Querying UDP session hardware entry created after receiving the first UDP packet:

    • Origin total bytes received

    • Origin total packets received

    • Reply total bytes received

    • Reply total packets received

Reference:

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp_query/flow_ct_udp_query_sample.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp_query/flow_ct_udp_query_main.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_udp_query/meson.build

Flow CT UDP Single Match

This sample is based on the Flow CT UDP sample. The sample illustrates that a hardware entry can be created with a single match (matching performed in one direction only) in the API call doca_flow_ct_add_entry().

Flow CT Aging

This sample illustrates the use of the DOCA Flow CT aging functionality. It demonstrates how to build a pipe and add different entries with different aging times and user data.

No packets need to be sent for this sample.

The sample logic includes:

  1. Initializing DOCA Flow by indicating mode_args="switch,hws" in the doca_flow_cfg struct.

  2. Initializing DOCA Flow CT .

  3. Starting two DOCA Flow uplink representor ports where port 0 has a special role of being a switch manager port.

    Info

    Ports are configured according to the parameters provided to doca_dpdk_port_probe() in the main function.

  4. Building a UDP pipe to serve as the root pipe.

  5. Building a counter pipe with an example 5-tuple entry to which CT forwards packets.

  6. Adding 32 entries with a different 5-tuple match, different aging time (3-12 seconds), and setting user data. User data will contain the port ID, entry number, and status.

  7. Handling aging in small intervals and removing each entry after age-out.

  8. Running these commands until all 32 entries age out.

Reference:

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_aging/flow_ct_aging_sample.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_aging/flow_ct_aging_main.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_aging/meson.build

Flow CT TCP Actions

This sample illustrates how a to add shared and non-shared actions to CT TCP sessions. The pipeline is identical to that of the Flow CT TCP sample.

Info

The sample expects to receive at least SYN and FIN packets.

This sample adds a shared action on one side of the session that placed the value 1 in the packet's metadata, while on the other side of the session a non-shared action is placed. The non-shared action simply flips the order of the source-destination IP addresses and port numbers.

The sample logic includes:

  1. Initializing DOCA Flow by indicating mode_args="switch,hws" in the doca_flow_cfg struct.

  2. Initializing DOCA Flow CT .

  3. Starting two DOCA Flow uplink representor ports where port 0 has a special role of being a switch manager port.

Info
  • Ports are configured according to the parameters provided to doca_dpdk_port_probe() in the main function.

  1. Creating a pipeline on the main port:

    1. Building an TCP pipe to filter non-TCP packets.

    2. Building a CT pipe to hold TCP session entries.

    3. Building a CT miss pipe which forwards all packets to RSS pipe.

    4. Building an RSS pipe from which all packets are directed to the sample main thread for parsing and processing.

    5. Building a TCP flags filter pipe which identifies the TCP flag inside the packets. SYN, FIN, and RST packets are forwarded the to RSS pipe while all others are forwarded to the EGRESS pipe.

    6. Building an EGRESS pipe to forward packets to uplink representor port 1.

  2. Packet processing:

    1. The first TCP packet triggers the miss flow as the CT pipe is empty.

    2. 5-tuple packet parsing is performed.

    3. TCP flag is examined.

      • In case of a SYN flag, a hardware entry is created.

      • For FIN or RST flags, the HW entry is removed and all packets are transferred to uplink representor port 1 using rte_eth_tx_burst()on port 0 (proxy port) by rte_flow_dynf_metadata_set() to 1.

    4. From this point on, all TCP packets belonging to the above session are offloaded directly to uplink port representor 1.

Reference:

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_tcp_actions/flow_ct_tcp_actions_sample.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_tcp_actions/flow_ct_tcp_actions_main.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_tcp_actions/meson.build

Flow CT TCP IPv4/IPv6

This sample illustrates how to manage a flow with a different IP type per direction.

In case of a SYN flag:

  1. A single HW entry of IPv4 is created as origin direction

  2. An additional HW entry of IPv6 is created as reply direction

  3. From this point on, all IP v4 TCP packets (belonging to the origin direction) and all IPv6 TCP packets (belonging to the reply direction) are offloaded.

Reference:

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_tcp/flow_ct_tcp_sample_ipv4_ipv6.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_tcp/flow_ct_tcp_ipv4_ipv6_main.c

  • /opt/mellanox/doca/samples/doca_flow/flow_ct_tcp/meson.build

© Copyright 2025, NVIDIA. Last updated on May 22, 2025.