DPL Service Configuration
The DPL Runtime Service supports has three types of configuration files.
These files follow a simplified ini-style format, allowing repeated sections (e.g., [INTERFACE]) and using # for comments instead of ;.
These are the available configuration files, refer to the next sections for details on each one:
DPL Device Configuration, found at
/etc/dpl_rt_service/devices.d/<device-id>.confService Daemon Configuration, found at
/etc/dpl_rt_service/dpl_rt.confSystem Configuration, found at
/etc/dpl_rt_service/system.conf
A default (recommended) daemon and system configuration files are provided and installed by the dpl_dpu_setup.sh script.
However, users must create their own DPL Device configuration file, based on a provided template config file (also installed by the dpl_dpu_setup.sh script).
Refer to the next section for detail about each config file.
This file defines the device, its interfaces, and how they map to DPL Port IDs used in DPL programs.
Path: /etc/dpl_rt_service/devices.d/<device-id>.conf (e.g., /etc/dpl_rt_service/devices.d/1000.conf)
A configuration template is available at /etc/dpl_rt_service/devices.d/NAME.conf.template.
If using SFs or SR-IOV VFs, make sure to reference their representor interfaces in the configuration file, not the SF/VF interfaces themselves.
DPL Port ID Mapping
Each physical or virtual interface is assigned a logical DPL Port ID (dpl_logical_port_id). You must ensure consistency between your DPL program and this configuration.
Supported interface types:
Uplink netdev interface (e.g.,
p0)PF representor (e.g.,
pf0hpf)VF representor (e.g.,
pf0vf0)SF representor (e.g.,
en3f0pf0sf1)
All listed interfaces must belong to the same uplink port (Unless multiport eswitch is enabled).
DPL Port ID Rules
The following rules and restrictions are applied to the dpl_device_id and dpl_logical_port_id values:
Condition | Requirement |
Reserved value |
|
DPL device ID | Must be a positive integer |
DPL interface ID | Integer between |
Number of uplink ports per config file | Only one. (Unless multiport eswitch is enabled). |
Example DPL Device Configuration File
# Example of a possible DPL RT Service Device configuration file:
#
# This configuration file specifies the DPL device and its interfaces
# and their DPL Port IDs that will be used by a DPL program.
#
# The DPL Port IDs are assigned by the user. The user decides which
# DPL Port ID is assigned to which ConnectX/DPU interface. This mapping
# is critical for achieving the desired results when adding table entries.
# For DPL device, the ID must be an integer number greater than zero.
#
# The configuration file consists of following sections:
# - [DEVICE] section: Must appear only once.
# - [P4_RT_CONTROLLER] section: Must appear only once.
# - [INTERFACE] section: Must be repeated for each DPL Port (network interface).
[DEVICE]
# The DPL Device ID, used for connecting a controller to manage this device's tables.
dpl_device_id=1000
# Cache counter - decrease HW accesses - when expired an HW access will occur upon request.
dpl_counter_cache_timeout=0
# Sets counter polling interval for idle-timeout. [seconds]
idle_timeout_polling_interval=2
# Polling interval for TCP state objects data from HW. [milli-seconds]
tcp_state_polling_interval_msec=200
[P4_RT_CONTROLLER]
# Packets delivered to the DPL RT Service from a controller will have this source DPL Port ID.
# So, this ID can be used for matching traffic originated from the controller.
p4_controller_port_id=9876
[INTERFACE]
# Interface name on the system to attach to this DPL device.
interface=p0
# DPL Port ID, used to reference this port by the DPL program and/or when updating table entries.
dpl_logical_port_id=0
# Ethernet frame size.
mtu=1514
# Only uncomment and provide this if you wish to override the interface's MAC address.
# mac=00:00:00:00:00:00
[INTERFACE]
interface=pf0hpf
dpl_logical_port_id=65535
mtu=1514
# mac=00:00:00:00:00:00
[INTERFACE]
interface=pf0vf0
dpl_logical_port_id=1
mtu=1514
# mac=00:00:00:00:00:00
[INTERFACE]
interface=pf0vf1
dpl_logical_port_id=2
mtu=1514
# mac=00:00:00:00:00:00
Section DEVICE
This section define the attributes of the DPL Device that consists of multiple ports and can be used for loading DPL program.
This section must appear exactly once in each device .conf file.
dpl_device_id
This is the Device ID value, which is used by controller applications for specifying the target device to connect to (for loading programs and managing them).
dpl_counter_cache_timeout
Cache counter - this is used for decreasing HW accesses done for reading counter. When expired an HW access will occur upon request.
Enabling this can result in provided outdated counter values.
idle_timeout_polling_interval
Sets counter polling interval for idle-timeout.
Defines the time interval between iterations of reading entries' counters from HW and checking which entries are stale (i.e. no traffic hit them for a period larger than each entry's defined idle timeout threshold).
This time interval affects both the Entry Timeout and the Delayed Counter Statistics. A large number will impact the delivery time accuracy of timeout notification or the refresh rate of counter statistics.
tcp_state_polling_interval_msec
Polling interval for TCP state objects data from HW in [milli-seconds].
Defines the interval between iterations done for reading TCP state objects from the HW.
Section P4_RT_CONTROLLER
This section provides additional configurations for communicating with P4 Controllers.
p4_controller_port_id
P4 controller applications can send packets (Packet Out) to the DPL RT Service using RPC messages.
Such packet will have a port ID equal to p4_controller_port_id.
The DPL programmer can check if a packet was originated from a P4 Controller by simply comparing the std_meta.ingress_port to the value of p4_controller_port_id in the device configuration file.
Section INTERFACE
This section defines the attributes of port that belongs to the device listed in DEVICE section.
This section can be repeated many times.
interface
Interface net-device name on the system.
dpl_logical_port_id
DPL Port ID, used to reference this port by the DPL program and/or when updating table entries.
Can be used for matching on ingress traffic on specific ports, and to send traffic to ports.
The DPL programmer can check if a packet was received on a specific port/interface by simply comparing the std_meta.ingress_port to dpl_logical_port_id value of the the corresponding interface in the device configuration file.
mtu
Defines the Ethernet frame size (MTU) for the device.
Currently, this field is ignored and has no effect.
mac
Defines the MAC address for the interface.
Currently, this field is ignored and has no effect.
Path: /etc/dpl_rt_service/dpl_rt.conf
This file configures core behavior for the DPL Runtime Service, including logging and gRPC server binding addresses.
# Example of a possible DPL RT Service GENERAL configuration file
[LOGGING]
log_file_path=/var/log/doca/dpl_rt_service/dpl_rtd.log
log_level=INFO
# Possible log_level values (case insensitive):
# DISABLE
# CRITICAL
# ERROR
# WARNING
# INFO
# DEBUG
# TRACE
[P4RT_RPC_SERVER]
server_address=[::] # IPv6 "ANY" allows IPv4 connections
server_tcp_port=9559
## If you would like to enable TLS authentication for the gRPC server connections, uncomment the following line and provided the required info:
# server_cert=/path/to/server.crt
# ca_cert=/path/to/ca.crt
# server_key=/path/to/server.key
[DPL_ADMIN_RPC_SERVER]
server_address=[::] # IPv6 "ANY" allows IPv4 connections
server_tcp_port=9600
## If you would like to enable TLS authentication for the gRPC server connections, uncomment the following line and provided the required info:
# server_cert=/path/to/server.crt
# ca_cert=/path/to/ca.crt
# server_key=/path/to/server.key
[DPL_NSPECT_RPC_SERVER]
server_address=[::] # IPv6 "ANY" allows IPv4 connections
server_tcp_port=9560
## If you would like to enable TLS authentication for the gRPC server connections, uncomment the following line and provided the required info:
# server_cert=/path/to/server.crt
# ca_cert=/path/to/ca.crt
# server_key=/path/to/server.key
[DPL_PACKET_IO]
enabled=true
[DPL_SHM]
enabled=true
Section LOGGING
log_file_path
Defines the path to the DPL RT Service log file.
log_level
Defines the default log level used after starting the DPL RT Service.
The following are the supported log levels (case insensitive):
DISABLE
CRITICAL
ERROR
WARNING
INFO
DEBUG
TRACE
You can dynamically change the logging level using the DPL Admin client, but this would not persist across restarts unless manually updated in the configuration file.
Section P4RT_RPC_SERVER
A gRPC server that listens for connections from P4Runtime clients. A sample open-source client is included in the DPL Dev container.
For details about the section parameters, refer to Common gRPC Servers Parameters below.
Section DPL_ADMIN_RPC_SERVER
A gRPC server that listens for connections
dpl_admin tool.
For details about the section parameters, refer to Common gRPC Servers Parameters below.
Section DPL_NSPECT_RPC_SERVER
A gRPC server that listen for connections from the DPL Debugger tools.
For details about the section parameters, refer to Common gRPC Servers Parameters below.
Common gRPC Servers Parameters
The following parameters are common for the P4RT_RPC_SERVER, DPL_ADMIN_RPC_SERVER and DPL_NSPECT_RPC_SERVER sections and have similar meaning.
server_address
The TCP binding address for the respective gRPC server.
You can specify server_address as [::] (IPv6 ANY) to allow connections from all interfaces, including IPv4.
Alternatively, you can restrict it to a specific IP address (e.g. a management interface).
server_tcp_port
The TCP binding port for the respective gRPC server.
If a custom non-default port is set in the configuration file, it must be used when connecting the corresponding client application as well.
TLS authentication support for the gRPC
A secured TLS authentication based connection can be supported and required by any of the gRPC servers separately from the other servers.
Each server has its own parameters for setting the required cert keys.
To enabled TLS authentication for a gRPC server connections, add all the following parameters to the corresponding server section in the config file.
These flags are m utually inclusive, either they are all set or unset.
server_cert
Path to server.crt certificate file.
ca_cert
Path to ca.cr certificate file.
server_key
Path to server.key certificate file.
Section DPL_PACKET_IO
enabled
When set to true, the DPL RT Service initializes the Packet IO infrastructure, which is required for support Packet IO from a P4 Controller, the Add Entry (from DPL program) feature and the DPL Debugger tools.
Section DPL_SHM
enabled
When set to true, the DPL RT Service initializes the Shared Memory (SHM) infrastructure, which is required for support a Controller application written using the DPL Runtime Controller SDK .
Path: /etc/dpl_rt_service/system.conf
This file tunes how the DPL Runtime Service interacts with hardware.
Example:
# Example of a possible DPL RT Service system configuration file
[LOGICAL_CORES]
# Logical CPU core for processing Packet IO (do not use lcore 0).
packet_io_lcores=1
# Logical CPU core for processing rule insertions/deletions (do not use lcore 0).
rule_insertion_lcores=2
[HAL]
# HWS queue size.
queue_size=1024
# Number of HWS queues for rule insertions/deletions.
queues_num=1
# Burst size for HWS rules insertions/deletions.
burst_size=32
Section LOGICAL_CORES
packet_io_lcores
Logical CPU core for processing Packet IO (do not use lcore 0).
rule_insertion_lcores
Logical CPU core for processing rule insertions/deletions (do not use lcore 0).
Must be different than cores used by a Controller application written using the DPL Runtime Controller SDK .
Section HAL
queue_size
HWS queue size.
Do not modify this settings unless instructed by NVIDIA Support. I directly impact performance characteristics such as rule update rate and latency.
queues_num
Number of HWS queues for rule insertions/deletions.
Do not modify this settings unless instructed by NVIDIA Support. I directly impact performance characteristics such as rule update rate and latency.
burst_size
Burst size for HWS rules insertions/deletions.
Do not modify this settings unless instructed by NVIDIA Support. I directly impact performance characteristics such as rule update rate and latency.