Template API

To speed up the rules insertion and avoid blocking the traffic, an additional API (functions and structs) was introduced to RTE_FLOW.

The template API introduces an approach in which applications can use rules templates in asynchronous manner enabling fast and efficient provisioning of rules. It is based on four key concepts:

1. Pre-configuration hints

   The application supplies some hints during the PMD configuration stage about the needed resources. This allows the PDM to pre-allocate all needed resources unlike the basic API in which the PMD needed to allocate the resources only at rule creation.

2. Flow grouping using templates

   The API uses templates to batch rules together. unlike the standard API which looked at each rule as a standalone rule. This allows faster insertion rate, as the PMD does not need to analyze each rule.

3. Queue-based flow management

   The API exposes the ability to add rules using different queues. This allows the application to insert rules without any locks (each queue must be accessed from single thread in this design).

4. Async insertion

   The new API supports async rule offloading. This allows the application to insert rules in async manner and only later check for completion.

5. Fine-grained application control

   This allows the PMD to be optimized and gives the application full control on the rule insertion.

Using the above concepts enables a much faster insertion rate along with minimum blocking of the application during the rule insertion/deletion.

DPDK upstream link: 13. Generic flow API (rte_flow) — Data Plane Development Kit 22.07.0 documentation (dpdk.org)

The template API extends the DPDK rte_flow API with the following DPDK functions:

• rte_flow_configure

• rte_flow_info_get

• rte_flow_pattern_template_create

• rte_flow_pattern_template_destroy

• rte_flow_actions_template_create

• rte_flow_actions_template_destroy

• rte_flow_template_table_create

• rte_flow_template_table_destroy

• rte_flow_async_create

• rte_flow_async_destroy

• rte_flow_push

• rte_flow_pull

• rte_flow_async_action_handle_create

• rte_flow_async_action_handle_destroy

• rte_flow_async_action_handle_update

• rte_flow_get_q_aged_flows

To enable template API, the application must enable it by setting dv_flow_en device argument to 2. This argument is a part of the device configuration in EAL command line arguments:

            

            
-a <pcie_bdf>,dv_flow_en=2
        

In the example below, the application is responsible for handling incoming traffic from two separate clients. Each client's traffic will be processed in software on a separate Rx queue:

• client A on Rx queue RX_QUEUE_ID_A

• entity B on Rx queue RX_QUEUE_ID_B

For the purpose of this example, we will assume that:

• traffic from client A will be either Ethernet/IPv4/UDP or Ethernet/IPv4/TCP and there will be at most 500k unique connections (i.e. 500k unique 5-tuples - IPv4 source and destination addresses, TCP/UDP, source and destination ports).

• traffic from client B will be either Ethernet/IPv4/UDP or Ethernet/IPv4/TCP and there will be at most 500k unique connections.

• sets of 5-tuples for client A and client B are disjoint, i.e. client can be uniquely identified by a 5-tuple.

As a result, the application should be able to process at most 1M unique connections and determine the destination queue based on 5-tuple. The following steps provide an example configuration of traffic processing pipeline supporting this use case. This pipeline consists of the following:

• Table, in group 1, containing at most 1M flows which:

  • match incoming traffic from client A (identified by a 5-tuple match), assign a TAG value 0xaaaa to the packet and transfer processing to group 4;

  • match incoming traffic from client B (identified by a 5-tuple match), assign a TAG value 0xbbbb to the packet and transfer processing to group 4;

• Table, in group 4, containing flows with:

  • match packets with TAG value 0xaaaa (traffic from client A) and forward traffic to RX_QUEUE_ID_A Rx queue.

  • match packets with TAG value 0xbbbb (traffic from client B) and forward traffic to RX_QUEUE_ID_B Rx queue.

Examples below assume that there is a flow create in group 0, which matches all traffic and transfers it for processing to group 1.

Step 1: Configure Template API

In this step application configures template API using rte_flow_configure(). In order to properly configure the template API, application must provide port configure (through rte_flow_port_attr struct) and queue configuration (through rte_flow_queue_attr struct).

Port configuration specifies the amount of resources to preallocate:

• counters

• aging objects

• meters

• meter profiles

• meter policies

• connection tracking objects

Before application attempts to create a flow with any of those resources, they must be preallocated at configuration time. Despite preallocating resources, application can provide additional flags to port configuration (flags field in rte_flow_port_attr struct). Currently the only supported flag is RTE_FLOW_PORT_FLAG_STRICT_QUEUE which controls strict queue mode:

• if set, all operations on a given flow (create/destroy/query/update) must happen on the same queue

• if unset, there is no limitation

Queue configuration specifies the size of flow queues, i.e. maximum number of flow queue operations allowed to be enqueued. Queue configuration is provided per queue.

Template API configuration can be done only if the port is not started.

In the examples below template API is configured without any additional resources and without enabling strict queue mode. Port will have 2 flow queues, each having 64 available entries.

            

            
struct rte_flow_error error = { 0 };
int ret;
 
uint16_t port_id = PORT_ID;
 
const struct rte_flow_port_attr port_flow_attr {
};
 
const struct rte_flow_queue_attr port_flow_q_attr = {
    .size = 64
};
const struct rte_flow_queue_attr *port_flow_q_attrs[] = {
    &port_flow_q_attr,
    &port_flow_q_attr,
};
 
/* If port is started, it must be stopped before configuration. */
rte_eth_dev_stop(port_id);
 
ret = rte_flow_configure(port_id, &port_flow_attr,
                         RTE_DIM(port_flow_q_attrs), port_flow_q_attrs,
                         &error);
if (ret != 0) {
    printf("Failed to configure template API: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
}
 
/* Port can be started now */
rte_eth_dev_start(port_id);
        

            

            
port stop 0
flow configure 0 queues_number 2 queues_size 64
port start 0
        

Step 2: Create Pattern Templates for Group 1

In this step pattern templates used in group 1 are created. Pattern template specify flow items (packet headers or packet metadata) on which flows will match processed packets. These template require specification of:

• Item types.

• Mask configuration of items:

  • If not provided, then flows will match only the given type, e.g. if RTE_FLOW_ITEM_TYPE_ETH is specified without a mask, then Ethernet packet will be matched.

  • If provided, then flows will match the given type and specified fields, e.g. RTE_FLOW_ITEM_TYPE_IPV4 is specified with a default mask (rte_flow_item_ipv4_mask), then IPv4 packets with specific source and destination addresses will be matched.

• Allowed direction attributes (ingress, egress and/or transfer).

• Whether to perform relaxed matching:

  • If relaxed matching is enabled, then matching is performed only on items with mask set.

  • If relaxed matching is disabled, then matching is performed on all items, regardless of mask configuration.

Examples below create 2 pattern templates:

1. Match packets with Ethernet, IPv4 and UDP headers. IPv4 source and destination addresses, and UDP source and destination ports will be matched.

2. Match packets with Ethernet, IPv4 and UDP headers. IPv4 source and destination addresses, and TCP source and destination ports will be matched.

   Source code

   testpmd commands

   Collapse Source Copy Copied! uint16_t port_id = PORT_ID;   const struct rte_flow_pattern_template_attr pt_attr = { .relaxed_matching = 0, .ingress = 1, }; struct rte_flow_error error = { 0 };   struct rte_flow_item pattern_1[] = { { .type = RTE_FLOW_ITEM_TYPE_ETH, }, { .type = RTE_FLOW_ITEM_TYPE_IPV4, .mask = &rte_flow_item_ipv4_mask, }, { .type = RTE_FLOW_ITEM_TYPE_UDP, .mask = &rte_flow_item_udp_mask, }, { .type = RTE_FLOW_ITEM_TYPE_END, }, }; struct rte_flow_pattern_template *pt_1;   pt_1 = rte_flow_pattern_template_create(port_id, &pt_attr, pattern_1, &error); if (pt_1 == NULL) { printf("Failed to create pattern template: %s (%d)\n", error.message, rte_errno); rte_exit(EXIT_FAILURE, "Exiting"); }   struct rte_flow_item pattern_2[] = { { .type = RTE_FLOW_ITEM_TYPE_ETH, }, { .type = RTE_FLOW_ITEM_TYPE_IPV4, .mask = &rte_flow_item_ipv4_mask, }, { .type = RTE_FLOW_ITEM_TYPE_TCP, .mask = &rte_flow_item_tcp_mask, }, { .type = RTE_FLOW_ITEM_TYPE_END, }, }; struct rte_flow_pattern_template *pt_2;   pt_2 = rte_flow_pattern_template_create(port_id, &pt_attr, pattern_2, &error); if (pt_2 == NULL) { printf("Failed to create pattern template: %s (%d)\n", error.message, rte_errno); rte_exit(EXIT_FAILURE, "Exiting"); }

   Copy Copied! flow pattern_template 0 create ingress relaxed no pattern_template_id 1 template eth / ipv4 src mask 255.255.255.255 dst mask 255.255.255.255 / udp src mask 0xffff dst mask 0xffff / end   flow pattern_template 0 create ingress relaxed no pattern_template_id 2 template eth / ipv4 src mask 255.255.255.255 dst mask 255.255.255.255 / tcp src mask 0xffff dst mask 0xffff / end

Step 3: Create Actions Template for Group 1

In this step an actions template is created. Actions template specify a list of actions performed on a packet when flow is matched. This template requires a specification of:

• Actions list - list of actions performed on a packet when flow is matched.

• Masks list - list of action masks specifying whether each flow performs the same action or not:

  • If corresponding action mask has a configuration (rte_flow_action struct mask field) and this configuration has non-zero fields, then each flow will perform the same action. Action parameters are based on action from the previous list. This configuration is named masked action.

  • If corresponding action mask does not have configuration or configuration is zeroed, then each flow will perform an action of this type, but action's configuration is taken from the flow rule. This configuration is named unmasked action.

• Allowed direction attribute (ingress, egress and/or transfer).

Examples below create an actions template which contains the following actions:

• MODIFY_FIELD action, which sets TAG with index 0 to a value provided in flow rule - action is unmasked.

• JUMP action, which transfer processing to group 4 - action is masked.

            

            
uint16_t port_id = PORT_ID;
struct rte_flow_error error = { 0 };
const struct rte_flow_actions_template_attr at_attr = {
    .ingress = 1,
};
 
struct rte_flow_action_modify_field set_tag_v = {
    .operation = RTE_FLOW_MODIFY_SET,
    .dst = {
        .field = RTE_FLOW_FIELD_TAG,
        .level = 0,
        .offset = 0,
    },
    .src = {
        .field = RTE_FLOW_FIELD_VALUE,
    },
    .width = 32,
};
struct rte_flow_action_modify_field set_tag_m = {
    .operation = RTE_FLOW_MODIFY_SET,
    .dst = {
        .field = RTE_FLOW_FIELD_TAG,
        .level = UINT32_MAX,
        .offset = UINT32_MAX,
    },
    .src = {
        .field = RTE_FLOW_FIELD_VALUE,
    },
    .width = UINT32_MAX,
};
*(uint32_t *)(&set_tag_v.src.value[0]) = 0;
*(uint32_t *)(&set_tag_m.src.value[0]) = 0;
 
struct rte_flow_action_jump jump_v = {
    .group = 4,
};
struct rte_flow_action_jump jump_m = {
    .group = UINT32_MAX,
};
 
struct rte_flow_action actions[] = {
    {
        .type = RTE_FLOW_ACTION_TYPE_MODIFY_FIELD,
        .conf = &set_tag_v,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_JUMP,
        .conf = &jump_v,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_END,
    },
};
struct rte_flow_action masks[] = {
    {
        .type = RTE_FLOW_ACTION_TYPE_MODIFY_FIELD,
        .conf = &set_tag_m,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_JUMP,
        .conf = &jump_m,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_END,
    },
};
struct rte_flow_actions_template *at;
 
at = rte_flow_actions_template_create(port_id, &at_attr,
                                      actions, masks, &error);
if (at == NULL) {
    printf("Failed to create actions template: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
}
        

            

            
flow actions_template 0 create ingress actions_template_id 1
    template modify_field op set
                          dst_type tag dst_level 0 dst_offset 0
                          src_type value src_value 00000000
                          width 32 /
             jump group 4 /
             end
    mask     modify_field op set
                          dst_type tag dst_level 0xffffffff
                                       dst_offset 0xffffffff
                          src_type value src_value 00000000
                          width 0xffffffff /
             jump group 0xffffffff /
             end
        

Step 4: Create Template Table for Group 1

In this step a template table is created. Template table requires a specification of common parameters of flows created in this table:

• flow group;

• flow prioriity;

• flow direction attribute (ingress, egress or transfer);

• list of pattern templates on which flow matching is based;

• list of actions templates on which flow actions are based;

• maximum number of flows allowed to be created in this table.

Examples below create a template table with the following configuration:

• group 1;

• priority 0 (highest priority);

• applicable to incoming traffic (ingress);

• flow matching is based on pattern templates created in step 1;

• flow actions are based on actions template created in step 2;

            

            
uint16_t port_id = PORT_ID;
struct rte_flow_error error = { 0 };
 
const struct rte_flow_template_table_attr tbl_attr = {
    .flow_attr = {
        .group = 1,
        .priority = 0,
        .ingress = 1,
    },
    .nb_flows = 1000000,
};
struct rte_flow_pattern_template *pts[] = { pt_1, pt_2 };
struct rte_flow_actions_template *ats[] = { at };
 
struct rte_flow_template_table *tbl;
 
tbl = rte_flow_template_table_create(port_id, &tbl_attr,
                                     pts, RTE_DIM(pts),
                                     ats, RTE_DIM(ats),
                                     &error);
if (tbl == NULL) {
    printf("Failed to create template table: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
}
        

            

            
flow template_table 0 create group 1 priority 0 ingress
    table_id 1 rules_number 1000000
    pattern_template 1 pattern_template 2
    actions_template 1
        

Step 5: Create Flows in Group 1 Table

In this step, flows in previously configured template table are created. To create a flow using template API, application must enqueue flow create operations onto flow queue. When enqueuing a flow create operation an application must provide:

• Port ID;

• Flow queue ID (should be less than a number of flow queues specified in a call to rte_flow_configure());

• Whether flow operation is postponed (postpone field of rte_flow_op_attr structure):

  • If operation is postponed, then it is not immediately processed by HW. To push an operation for processing, application must call rte_flow_push() which will flush queued operations to HW;

  • If operation is not postponed, then the command (and previously postponed commands) will be pushed to HW for processing;

• Handle to previously created template table;

• List of pattern items:

  • If a corresponding item in pattern template has a mask specified, then pattern item should specify matching value (through spec field of rte_flow_item structure).

  • Otherwise, only the item type is required.

• Index of pattern template on which flow is based. Index is an offset of the template in a list provided on template create creation.

• List of actions:

  • If a corresponding action in actions template is unmasked, then action configuration must be provided.

  • Otherwise, only the action type is required.

• Pointer to application's data related to the flow (through user_data parameter).

  • This data will be provided as a part of a flow operation completion. It can be helpful in, e.g. identifying a flow.

After flow create operation is enqueued and pushed to HW, application must poll for flow operation completion using rte_flow_pull() call. The result of the operation will be stored in provided array of rte_flow_op_result structures.

Examples below create 2 flows:

1. Flow which matches Ethernet, IPv4 and UDP traffic. IPv4 source and destination address must be, respectively, 10.0.0.1 and 10.0.0.2. UDP source and destination ports must be, respectively, 10001 and 10002. This traffic comes from client A.

   When matched, TAG will be set to 0xaaaa and processing will be transferred to group 4 (masked JUMP action).

2. Flow which matches Ethernet, IPv4 and TCP traffic. IPv4 source and destination address must be, respectively, 10.0.0.1 and 10.0.0.2. TCP source and destination ports must be, respectively, 20001 and 20002. This traffic comes from client B.

   When matched, TAG will be set to 0xbbbb and processing will be transferred to group 4 (masked JUMP action).

            

            
uint16_t port_id = PORT_ID;
uint32_t queue_id = QUEUE_ID;
 
struct rte_flow_op_attr attr = {
    .postpone = 0,
};
struct rte_flow_item_ipv4 ipv4_spec = {
    .hdr = {
        .src_addr = RTE_BE32(RTE_IPV4(10, 0, 0, 1)),
        .dst_addr = RTE_BE32(RTE_IPV4(10, 0, 0, 2)),
    },
};
struct rte_flow_item_udp udp_spec = {
    .hdr = {
        .src_port = RTE_BE16(10001),
        .dst_port = RTE_BE16(10002),
    },
};
struct rte_flow_item pattern_1[] = {
    {
        .type = RTE_FLOW_ITEM_TYPE_ETH,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_IPV4,
        .spec = &ipv4_spec,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_UDP,
        .spec = &udp_spec,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_END,
    },
};
struct rte_flow_action_modify_field set_tag_1 = {
    .operation = RTE_FLOW_MODIFY_SET,
    .dst = {
        .field = RTE_FLOW_FIELD_TAG,
    },
    .src = {
        .field = RTE_FLOW_FIELD_VALUE,
    },
    .width = 32,
};
*(uint32_t *)(&set_tag_1.src.value[0]) = 0x0000aaaa;
struct rte_flow_action actions_1[] = {
    {
        .type = RTE_FLOW_ACTION_TYPE_MODIFY_FIELD,
        .conf = &set_tag_1,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_JUMP,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_END,
    },
};
struct rte_flow_error error = { 0 };
struct rte_flow *flow_1;
 
flow_1 = rte_flow_async_create(port_id, queue_id, &attr, tbl,
                               pattern_1, 0, actions_1, 0,
                               NULL, &error);
if (flow_1 == NULL) {
    printf("Failed to enqueue flow create operation: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
};
 
while (true) {
    struct rte_flow_op_result res = { 0 };
    int ret;
 
    ret = rte_flow_pull(port_id, queue_id, &res, 1, &error);
    if (ret < 0) {
        printf("Failed to pull flow results: %s (%d)\n", error.message, rte_errno);
        rte_exit(EXIT_FAILURE, "Exiting");
    } else if (ret == 1) {
        if (res.status == RTE_FLOW_OP_ERROR) {
            printf("Flow create operation failed\n");
            rte_exit(EXIT_FAILURE, "Exiting");
        }
        break;
    }
}
 
// Inserting tcp flow
struct rte_flow_item_tcp tcp_spec = {
    .hdr = {
        .src_port = RTE_BE16(20001),
        .dst_port = RTE_BE16(20002),
    },
};
struct rte_flow_item pattern_2[] = {
    {
        .type = RTE_FLOW_ITEM_TYPE_ETH,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_IPV4,
        .spec = &ipv4_spec,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_TCP,
        .spec = &tcp_spec,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_END,
    },
};
struct rte_flow_action_modify_field set_tag_2 = {
    .operation = RTE_FLOW_MODIFY_SET,
    .dst = {
        .field = RTE_FLOW_FIELD_TAG,
    },
    .src = {
        .field = RTE_FLOW_FIELD_VALUE,
    },
    .width = 32,
};
*(uint32_t *)(&set_tag_2.src.value[0]) = 0x0000bbbb;
struct rte_flow_action actions_2[] = {
    {
        .type = RTE_FLOW_ACTION_TYPE_MODIFY_FIELD,
        .conf = &set_tag_2,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_JUMP,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_END,
    },
};
struct rte_flow *flow_2;
 
flow_2 = rte_flow_async_create(port_id, queue_id, &attr, tbl,
                               pattern_2, 1, actions_2, 0,
                               NULL, &error);
if (flow_2 == NULL) {
    printf("Failed to enqueue flow create operation: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
};
 
/* Poll for flow operation completion like for a previous flow. */
        

            

            
flow queue 0 create 0 template_table 1 pattern_template 0 actions_template 0 postpone no
    pattern eth /
            ipv4 src spec 10.0.0.1 dst spec 10.0.0.2 /
            udp src spec 10001 dst spec 10002 /
            end
    actions modify_field op set dst_type tag src_type value src_value aaaa0000 width 32 /
            jump /
            end
flow pull 0 queue 0
 
flow queue 0 create 0 template_table 1 pattern_template 1 actions_template 0 postpone no
    pattern eth /
            ipv4 src spec 10.0.0.1 dst spec 10.0.0.2 /
            tcp src spec 20001 dst spec 20002 /
            end
    actions modify_field op set dst_type tag src_type value src_value bbbb0000 width 32 /
            jump /
            end
flow pull 0 queue 0
        

Step 6: Create Pattern Template for Group 4

In this step, a pattern template used in group 4 is created. This template specifies that packets will be matched only against a value of the TAG with index 0.

            

            
uint16_t port_id = PORT_ID;
struct rte_flow_error error = { 0 };
const struct rte_flow_pattern_template_attr pt_4_attr = {
    .relaxed_matching = 0,
    .ingress = 1,
};
 
struct rte_flow_item_tag tag_spec = {
    .index = 0,
};
struct rte_flow_item pattern_4[] = {
    {
        .type = RTE_FLOW_ITEM_TYPE_TAG,
        .spec = &tag_spec,
        .mask = &rte_flow_item_tag_mask,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_END,
    },
};
struct rte_flow_pattern_template *pt_4;
 
pt_4 = rte_flow_pattern_template_create(port_id, &pt_4_attr, pattern_4, &error);
if (pt_4 == NULL) {
    printf("Failed to create pattern template: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
}
        

            

            
flow pattern_template 0 create ingress relaxed no pattern_template_id 4
    template tag data mask 0xffffffff index is 0 / end
        

Step 7: Create Actions Template for Group 4

In this step, an actions template used in group 4 is created. This template specifies a single unmasked QUEUE action. This action, when performed, will forward packets to the Rx queue specified in flow rule.

            

            
uint16_t port_id = PORT_ID;
struct rte_flow_error error = { 0 };
const struct rte_flow_actions_template_attr at_attr = {
    .ingress = 1,
};
 
struct rte_flow_action actions_4[] = {
    {
        .type = RTE_FLOW_ACTION_TYPE_QUEUE,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_END,
    },
};
struct rte_flow_action masks_4[] = {
    {
        .type = RTE_FLOW_ACTION_TYPE_QUEUE,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_END,
    },
};
struct rte_flow_actions_template *at_4;
 
at_4 = rte_flow_actions_template_create(port_id, &at_attr,
                                        actions_4, masks_4, &error);
if (at_4 == NULL) {
    printf("Failed to create actions template: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
}
        

            

            
flow actions_template 0 create ingress actions_template_id 4
    template queue / end
    mask     queue / end
        

Step 8: Create Template Table for Group 4

In this step, a template table in group 4 is created. This table has a following configuration:

• group 1;

• priority 0 (highest priority);

• applicable to incoming traffic (ingress);

• flow matching is based on pattern template created in step 5;

• flow actions are based on actions template created in step 6;

            

            
uint16_t port_id = PORT_ID;
struct rte_flow_error error = { 0 };
 
const struct rte_flow_template_table_attr tbl_attr = {
    .flow_attr = {
        .group = 4,
        .priority = 0,
        .ingress = 1,
    },
    .nb_flows = 1000000,
};
struct rte_flow_pattern_template *pts[] = { pt_4 };
struct rte_flow_actions_template *ats[] = { at_4 };
 
struct rte_flow_template_table *tbl_4;
 
tbl_4 = rte_flow_template_table_create(port_id, &tbl_attr,
                                       pts, RTE_DIM(pts),
                                       ats, RTE_DIM(ats),
                                       &error);
if (tbl_4 == NULL) {
    printf("Failed to create template table: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
}
        

            

            
flow template_table 0 create group 4 priority 0 ingress
    table_id 4 rules_number 1000000
    pattern_template 4
    actions_template 4
        

Step 9: Create Flows in Group 4 Table

In this step, 2 flows are created in table from step 7:

1. Flow which matches on TAG (index 0) value of 0xaaaa. If matched, traffic will be forwarded to RX_QUEUE_ID_A Rx queue.

2. Flow which matches on TAG (index 0) value of 0xbbbb. If matched, traffic will be forwarded to RX_QUEUE_ID_B Rx queue.

            

            
uint16_t port_id = PORT_ID;
uint32_t queue_id = QUEUE_ID;
 
struct rte_flow_op_attr attr = {
    .postpone = 0,
};
struct rte_flow_item_tag tag_spec_1 = {
    .data = 0x0000aaaa,
    .index = 0,
};
struct rte_flow_item_tag tag_mask_1 = {
    .data = 0xffffffff,
    .index = 0xff,
};
struct rte_flow_item pattern_4_1[] = {
    {
        .type = RTE_FLOW_ITEM_TYPE_TAG,
        .spec = &tag_spec_1,
        .mask = &tag_mask_1,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_END,
    },
};
struct rte_flow_action_queue queue_rxq_A = {
    .index = RX_QUEUE_ID_A,
};
struct rte_flow_action actions_4_1[] = {
    {
        .type = RTE_FLOW_ACTION_TYPE_QUEUE,
        .conf = &queue_rxq_A,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_END,
    },
};
struct rte_flow_error error = { 0 };
struct rte_flow *flow_4_1;
 
flow_4_1 = rte_flow_async_create(port_id, queue_id, &attr, tbl_4,
                                 pattern_4_1, 0, actions_4_1, 0,
                                 NULL, &error);
if (flow_4_1 == NULL) {
    printf("Failed to enqueue flow create operation: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
};
 
/* Poll for flow operation completion - please see step 4. */
 
struct rte_flow_item_tag tag_spec_2 = {
    .data = 0x0000bbbb,
    .index = 0,
};
struct rte_flow_item_tag tag_mask_2 = {
    .data = 0xffffffff,
    .index = 0xff,
};
struct rte_flow_item pattern_4_2[] = {
    {
        .type = RTE_FLOW_ITEM_TYPE_TAG,
        .spec = &tag_spec_2,
        .mask = &tag_mask_2,
    },
    {
        .type = RTE_FLOW_ITEM_TYPE_END,
    },
};
struct rte_flow_action_queue queue_rxq_B = {
    .index = RX_QUEUE_ID_B,
};
struct rte_flow_action actions_4_2[] = {
    {
        .type = RTE_FLOW_ACTION_TYPE_QUEUE,
        .conf = &queue_rxq_B,
    },
    {
        .type = RTE_FLOW_ACTION_TYPE_END,
    },
};
 
struct rte_flow *flow_4_2;
flow_4_2 = rte_flow_async_create(port_id, queue_id, &attr, tbl_4,
                                 pattern_4_2, 0, actions_4_2, 0,
                                 NULL, &error);
if (flow_4_2 == NULL) {
    printf("Failed to enqueue flow create operation: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting");
};
 
/* Poll for flow operation completion - please see step 4. */
        

            

            
# It is assumed that RX_QUEUE_ID_A == 0
flow queue 0 create 0 template_table 4 pattern_template 0 actions_template 0 postpone no
    pattern tag data is 0x0000aaaa index is 0 / end
    actions queue index 0 / end
flow pull 0 queue 0
 
# It is assumed that RX_QUEUE_ID_A == 1
flow queue 0 create 0 template_table 4 pattern_template 0 actions_template 0 postpone no
    pattern tag data is 0x0000bbbb index is 0 / end
    actions queue index 1 / end
flow pull 0 queue 0
        

Destroying flows is similar to creating flows, application must enqueue a flow destroy operation on a flow queue (providing a flow handle returned by rte_flow_async_create()) and poll for operation's completion.

            

            
uint16_t port_id = PORT_ID;
uint32_t queue_id = QUEUE_ID;
struct rte_flow *flow = /* Flow handle returned from rte_flow_async_create(). */;
 
struct rte_flow_error error = { 0 };
int ret;
 
struct rte_flow_op_attr attr = {
    .postpone = 0,
};
 
ret = rte_flow_async_destroy(port_id, queue_id, &attr, flow, NULL, &error);
if (ret != 0) {
    printf("Failed to enqueue flow destroy operation: %s (%d)\n", error.message, rte_errno);
    rte_exit(EXIT_FAILURE, "Exiting"); 
}
 
/* Poll for destroy operation's completion. */
while (true) {
    struct rte_flow_op_result res = { 0 };
    int ret;
 
    ret = rte_flow_pull(port_id, queue_id, &res, 1, &error);
    if (ret < 0) {
        printf("Failed to pull flow results: %s (%d)\n", error.message, rte_errno);
        rte_exit(EXIT_FAILURE, "Exiting");
    } else if (ret == 1) {
        if (res.status == RTE_FLOW_OP_SUCCESS) {
            printf("Flow destroy operation succeeded\n");
        } else {
            printf("Flow destroy operation failed\n");
            rte_exit(EXIT_FAILURE, "Exiting"); 
        }
        break;
    }
}
        

            

            
# After enqueuing a flow, testpmd prints out a flow ID in "flow rule #ID" format
testpmd> flow queue 0 create 0 template_table 4 pattern_template 0 actions_template 0 postpone no
            pattern tag data is 0x0000aaaa index is 0 / end
            actions queue index 0 / end
Flow rule #0 creation enqueued
testpmd> flow pull 0 queue 0
Queue #0 pulled 1 operations (0 failed, 1 succeeded)
 
# To destroy this flow, provide the flow's ID to flow queue <port> destroy <queue> rule <rule-id> command
testpmd> flow queue 0 destroy 0 rule 0
Flow rule #0 destruction enqueued
testpmd> flow pull 0 queue 0
Queue #0 pulled 1 operations (0 failed, 1 succeeded)
        

Create flow with connection tracking action.

            

            
/* Indirect action create. */
struct rte_flow_action_handle *ct_handle;
struct rte_flow_action action;
struct rte_flow_action_conntrack pro = {0};
struct rte_flow_indir_action_conf indir_action_conf = { .ingress = 1, };
 
pro.enable = 1;
...
pro.is_original_dir = 1;
 
action.type = RTE_FLOW_ACTION_TYPE_CONNTRACK;
action.conf = &pro;
 
ct_handle = rte_flow_action_handle_create(port_id, &conf, &action, error);
 
/* Async indirect action create. */
struct rte_flow_action_handle *ct_handle
struct rte_flow_op_attr op_attr;
struct rte_flow_action action;
struct rte_flow_action_conntrack pro = {0};
struct rte_flow_indir_action_conf conf = { .ingress = 1, };
 
op_attr.postpone = 0; // or 1
 
pro.enable = 1;
...
pro.is_original_dir = 1;
 
action.type = RTE_FLOW_ACTION_TYPE_CONNTRACK;
action.conf = &pro;
 
ct_handle = rte_flow_async_action_handle_create(port_id, queue_id, &op_attr,
							&conf, &action, user_data, error);					
rte_flow_push(port_id, queue_id, error);
ret = rte_flow_pull(port_id, queue_id, res_n, res[], error);
for (i = 0; i < ret; i++)
	if (res[i].user_data == user_data)
		/* CT handle available now. */
 
/* Pattern template create. */
...
 
/* Action template create. */
actions[i].type = RTE_FLOW_ACTION_TYPE_INDIRECT;
actions[i].conf = NULL;
 
masks[i].type = RTE_FLOW_ACTION_TYPE_CONNTRACK;
masks[i].conf = NULL;
 
pattern_template = rte_flow_actions_template_create(port_id, attr, actions, masks, error);
 
/* Table create. */
...
/* Flow create. */
actions[i].type = RTE_FLOW_ACTION_TYPE_INDIRECT;
actions[i].conf = ct_handle;
 
flow = rte_flow_async_create(port_id, queue_id, op_attr, table, 
							 pattern, pattern_template_index,
							 actions, action_template_index,
							 user_data, error);
        

Enables the user to identify the IPv6 routing extension header's presence and match the 1st 32bits. In testpmd CLI, the user can specify the desired encapsulation pattern of IPv6 routing extension for raw_encap. raw_decap will remove it automatically if present.

Testpmd CLI Examples

• To match the 1st 32bits, run:

  flow pattern_template 0 create transfer relaxed no pattern_template_id 1 template represented_port ethdev_port_id is 0 / eth / ipv6 / ipv6_routing_ext ext_next_hdr is 17 is 2 ext_type is 4 / udp / end

• To encapsulate the pattern of IPv6 routing extension for raw_encap, run:

  set raw_encap eth src is 10:22:33:44:55:60 dst is a0:bb:cc:dd:ee:f2 / ipv6 src is 5::5 dst is 6::6 hop is 64 / ipv6_routing_ext ext_type is 4 ext_next_hdr is 41 ext_seg_left is 1 / end_set

• To modify the IPv6 protocol, run:

  flow actions_template 0 create transfer actions_template_id 3 template modify_field op set dst_type ipv6_proto src_type value src_value 0x11 width 8 / jump group 1 / end mask modify_field op set dst_type ipv6_proto dst_level 0xffffffff dst_offset 0xffffffff src_type value src_value 0x12345678 width 0xffffffff / jump group 1 / end

            

            
uint16_t port_id = 0;
const struct rte_flow_pattern_template_attr pt_attr = {
        .relaxed_matching = 0,
        .ingress = 1,
};
const struct rte_flow_item_ipv6_routing_ext mask = {
        .hdr =  {
                .next_hdr = 0xff;
                .type = 0xff;
                .segments_left = 0xff;
        },
};
struct rte_flow_error error = { 0  };
struct rte_flow_item pattern_1[] = {
        {
                .type = RTE_FLOW_ITEM_TYPE_ETH,
        },
        {
                .type = RTE_FLOW_ITEM_TYPE_IPV6,
        },
        {
                .type = RTE_FLOW_ITEM_TYPE_IPV6_ROUTING_EXT,
                .mask = &mask,
        },
        {
                .type = RTE_FLOW_ITEM_TYPE_END,
        },
 
};
struct rte_flow_pattern_template *pt_1;
pt_1 = rte_flow_pattern_template_create(port_id, &pt_attr, pattern_1, &error);
        

Overview

In this release, PMD supports pushing/removing IPv6 extensions into/from the packets. For now, the IPPROTO_ROUTING is the only supported extension type.

Testpmd commands

Remove action

            

            
> set ipv6_ext_remove 1 ipv6_ext type is 43 / end_set
 
> flow pattern_template 0 create transfer relaxed no pattern_template_id 1 template represented_port ethdev_port_id is 0 / eth / ipv6 / ipv6_routing_ext / udp src is 100 / end
 
> flow actions_template 0 create transfer actions_template_id 1 template ipv6_ext_remove index 1 / represented_port ethdev_port_id 1 / end mask ipv6_ext_remove index 1 / represented_port ethdev_port_id 1 / end
 
> flow template_table 0 create group 0 priority 0 transfer table_id 1 rules_number 128 pattern_template 1 actions_template 1
        

The remove action is unified in the template table and the action must be fully masked in the action template.

Push Action

            

            
> set ipv6_ext_push 1 ipv6_ext type is 43 / ipv6_routing_ext ext_type is 4 ext_next_hdr is 17 ext_seg_left is 2 / end_set
 
> flow pattern_template 0 create transfer relaxed no pattern_template_id 1 template represented_port ethdev_port_id is 0 / eth / ipv6 / udp src is 1 / end
 
> flow actions_template 0 create transfer actions_template_id 1 template ipv6_ext_push index 1 / represented_port ethdev_port_id 1 / end mask ipv6_ext_push index 1 / represented_port ethdev_port_id 1 / end
 
> flow template_table 0 create group 0 priority 0 transfer table_id 1 rules_number 128 pattern_template 1 actions_template 1
        

DPDK Code Snippets

Remove action

            

            
const struct rte_flow_action_ipv6_ext_remove remove_v = { .type = IPPROTO_ROUTING };
 
const struct rte_flow_action_ipv6_ext_remove remove_m = { .type = UINT8_MAX };
 
const struct rte_flow_action_queue queue_v = { .index = 0 };
 
const struct rte_flow_action_queue queue_m = { .index = UINT16_MAX };
 
const struct rte_flow_action actions[3] = {
 
    [0] = {
 
            .type = RTE_FLOW_ACTION_TYPE_IPV6_EXT_REMOVE,
 
            .conf = &remove_v,
 
},
 
[1] = {
 
            .type = RTE_FLOW_ACTION_TYPE_QUEUE,
 
            .conf = &queue_v,
 
    },
 
    [2] = { .type = RTE_FLOW_ACTION_TYPE_END }
 
};
 
const struct rte_flow_action masks[3] = {
 
    [0] = {
 
            .type = RTE_FLOW_ACTION_TYPE_IPV6_EXT_REMOVE,
 
            .conf = &remove_m,
 
},
 
[1] = {
 
            .type = RTE_FLOW_ACTION_TYPE_QUEUE,
 
            .conf = &queue_m,
 
    },
 
    [2] = { .type = RTE_FLOW_ACTION_TYPE_END }
 
};
 
 
 
 
const struct rte_flow_actions_template_attr at_attr = { .ingress = 1 };
 
rte_flow_actions_template_create(port_id, &at_attr, actions, masks, error);
        

Push action

            

            
const struct rte_flow_action_ipv6_ext_push push_v = { .type = IPPROTO_ROUTING, .size = 40, .data = {0x11, 4, 4, 2, 1, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4} };
 
const struct rte_flow_action_queue queue_v = { .index = 0 };
 
const struct rte_flow_action_queue queue_m = { .index = UINT16_MAX };
 
const struct rte_flow_action actions[3] = {
 
    [0] = {
 
            .type = RTE_FLOW_ACTION_TYPE_IPV6_EXT_PUSH,
 
            .conf = &push_v,
 
},
 
[1] = {
 
            .type = RTE_FLOW_ACTION_TYPE_QUEUE,
 
            .conf = &queue_v,
 
    },
 
    [2] = { .type = RTE_FLOW_ACTION_TYPE_END }
 
};
 
const struct rte_flow_action masks[3] = {
 
    [0] = {
 
            .type = RTE_FLOW_ACTION_TYPE_IPV6_EXT_PUSH,
 
            .conf = NULL,
 
},
 
[1] = {
 
            .type = RTE_FLOW_ACTION_TYPE_QUEUE,
 
            .conf = &queue_m,
 
    },
 
    [2] = { .type = RTE_FLOW_ACTION_TYPE_END }
 
};
 
 
 
 
const struct rte_flow_actions_template_attr at_attr = { .ingress = 1 };
 
rte_flow_actions_template_create(port_id, &at_attr, actions, masks, error);
        

General

1. The application must select one of the API at the start of the application by using device parameters flow_dv_en = 1(non template API) or 2 (template API).

   During the application run time, it can only use the API selected at startup. If non template API was selected, the application cannot use any of the template functions (listed in the API section of this document).

   If application selected the template API, it cannot use the rte_flow_create and rte_flow_destroy. It is recommended to use the async handle creation/query/update/destroy over the sync ones when using template API.

2. When configuring the PMD to work with the template API, the application cannot use the old one.

3. Not all matching and actions are supported using the template API. Please see the support matrix below for further information.

4. PMD is working in isolated mode which means that in order to get traffic, the application must manually add rules to get the matching traffic.

5. When creating rules with partial matching, while using template with partial mask (the mask value of the item field is not set to all bitwise ones), the spec (value) of the field (rule instance), should be set to field_to_set=field value & mask. (all bits in the value that are masked out must be zeroed)

6. The number of fields that are used in the matching patterns, when used in group > 0 or in FDB (transfer), is limited (compared to group = 0).

   Note: When exceeding this size pattern, template will fail.

7. There is a fixed action order:

   1. ingress: mark / decap / pop_vlan / modify_field / counter / meter / encap / jump / queue / rss / drop

   2. egress: counter / push_vlan / modify_field / meter / ecnap / jump / drop

   3. transfer: decap / pop_vlan / counter / meter / encap / jump / represented_port / drop /

8. In NVIDIA ConnectX-6 Dx/ConnectX-7/BlueField-2, the template API cannot match both IPv6 source and destination in the same rule.

9. The number of modified fields in one rule is 8. (Modify IPv6 source require for example 4 commands)

Supported Scale (also true for non template API)

1. The total number of header reformat (encap/decap) is 16M per DPDK port (PF/VF).

2. The total number of counters, + aging is 16M per DPDK port (PF/VF).

3. The total number of meters is 16M per DPDK port (PF/VF).

With the introduction of the template API we can now offer a significant improvement in rules insertion. The template API is about to support all existing DPDK items that are currently supported in the non template API.

Future DPDK items are planned to be supported through this API only. At this point, the template API supports most of the existing DPDK items but not all of them.

This table shows the supported item matrix between template and non template API

This table shows the supported action matrix between template and non-template API.