DPL Runtime Controller SDK
DPL applications are deployed to the NVIDIA® BlueField® networking platform (DPU or SuperNIC) using the P4Runtime API. However, tables annotated with nv_high_update_rate=true are managed exclusively through the dpl_rt_controller SDK.
Control Protocols
The DPL runtime service implements two control plane protocols for managing data plane elements:
P4Runtime – Based on the open, gRPC-based standard (P4Runtime Spec)
NVIDIA Proprietary Protocol – Optimized for High Update Rate tables, this protocol operates over shared memory.
P4Runtime controllers can run remotely on systems with TCP/IP access to the DPU where the dpl_rt_service container is active. The proprietary dpl_rt_controller must run locally on the DPU, as it attaches to shared memory created by the dpl_rt_service container.
The proprietary dpl_rt_controller SDK provides two controller interfaces:
dpl_p4rt_controller(gRPC-based) – Used to manage all data plane elements:Configure pipeline (load P4 program)
Manage table entries (add/delete, read counters, read entries)
Perform packet I/O (send/receive packets from the P4Runtime server)
Receive idle timeout notifications (for tables supporting timeouts)
dpl_rt_controller(Shared Memory-based) – Used to control High Update Rate (HUR) table entries:Add/delete operations
Counter updates
Limited to tables marked with
nv_high_update_rate=true
The dpl_rt_controller can be optionally integrated with a P4Runtime controller to offer unified table management across both regular and high-update-rate tables.
Table Types
The DOCA Pipeline Language (DPL) supports two categories of tables:
Regular P4 Table
Defined per standard P4Runtime specifications.
Managed via the gRPC-based P4Runtime controller.
Supports all match types supported by the DPL compiler.
High Update Rate Table
Declared like a regular table, with the
nv_high_update_rate = trueattribute. See High Update Rate Tables in the DOCA Target Architecture.Managed via the proprietary
dpl_rt_controllerSDK.Designed for optimized performance:
Eliminates gRPC overhead.
Avoids protobuf encoding.
Uses rule handles instead of hash-table lookups.
Restrictions:
Only exact match is supported; this is enforced by the DPL Compiler.
Inserting duplicate entries is not rejected by
dpl_rtd. It is the responsibility of the user’s controller application to avoid inserting duplicates.Each duplicate entry will receive a distinct
dpl_rt_controller_entryhandle. However, previously existing handles will be invalidated.Deleting an invalidated handle will trigger a completion error. This error is silently ignored by
dpl_rtdbut is tracked in thedpl_rt_controller_table_statistics.deletion_errorscounter for debugging purposes.
HUR tables cannot have constant entries defined in the DPL program.
HUR tables are only manageable via the proprietary
dpl_rt_controllerSDK:No API for entry lookup. The user controller application must maintain its own map between rules and their
dpl_rt_controller_entryhandles.HUR tables do not support the use of gRPC-based P4Runtime controllers for adding, deleting, or reading entries.
Non HUR tables can continue to use P4Runtime for management.
SDK Libraries Overview
The DPL SDK includes the following components:
Header Files
Path | Purpose |
| gRPC-based P4Runtime controller |
| Proprietary controller for High Update Rate tables |
Shared Libraries
Library | Description |
| Auto-generated P4Runtime |
| gRPC implementation for P4Runtime |
| API implementation for shared-memory-based High Update Rate tables |
Sample Applications
Sample controller applications can be found under:
/opt/dpl_rt_controller/samples/
Currently, only the Ubuntu 22.04 operating system is supported.
The DPL Runtime Controller SDK packages are provided via the dpl_rt_service NGC Resource package. For more information, refer to DPL Container Deployment, section "Pulling the Container Resources and Scripts from NGC" and perform the "Installing the DPL Runtime Controller SDK packages" step.
When loading DPL programs with High Update Rate tables, HugePages are used for pre-allocating entries buffers.
The required entry buffers size depends on the table properties (e.g. number of keys and table size).
The system preparation script dpl_dpu_setup.sh allocates 2048 HugePages by default.
This might not be enough for some programs, in such cases the dpl_rtd fails loading the program and logs an error containing the (Try increasing HugePages number) message.
To avoid such issues, run the dpl_dpu_setup.sh script with an appropriate --hugepages-num value.
For example, for loading a HUR table of 16 million entries size, run sudo <PATH TO>/dpl_dpu_setup.sh --hugepages-num 4096
APIs:
namespace DPL_P4RT_Controller {
/* -----------------------------------------------------------------------
* Controller class for connecting to dpl_rtd over gRPC.
* ----------------------------------------------------------------------- */
class Controller {
public:
/*
* -----------------------------------------------------------------------
* gRPC APIs
* -----------------------------------------------------------------------
*/
/**
* @brief Construct a new Controller object.
*
* Assigns default election IDs of (high = 0, low = 1).
*
* @param [in] device_id
* The Device ID to connect to in the dpl_rtd.
* @param [in] ipaddr
* The address for connecting to dpl_rtd
*/
DOCA_EXPERIMENTAL
Controller(uint32_t device_id, const char *ipaddr);
/**
* @brief Construct a new Controller object with custom election IDs.
*
* @param [in] device_id
* The Device ID to connect to in the dpl_rtd.
* @param [in] ipaddr
* The address for connecting to dpl_rtd
* @param [in] election_id_high
* High part of the 128bit election ID used to determine the primary controller.
* @param [in] election_id_low
* Low part of the 128bit election ID used to determine the primary controller.
*/
DOCA_EXPERIMENTAL
Controller(uint32_t device_id, const char *ipaddr, uint64_t election_id_high, uint64_t election_id_low);
/**
* @brief Destroy the Controller object
*
*/
DOCA_EXPERIMENTAL
~Controller();
/**
* @brief Create an unsecured connection to dpl_rtd.
*
* This requires that TLS authentication was not enabled on dpl_rtd P4RT_RPC_SERVER.
*
* Uses grpc::InsecureChannelCredentials().
*
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t ConnectUnsecured(void);
/**
* @brief Create a secured TLS connection to dpl_rtd.
*
* This requires that TLS authentication was properly enabled and configured on dpl_rtd P4RT_RPC_SERVER.
*
* Uses grpc::SslCredentials().
*
* @param [in] ca_crt_path
* Path to ca.crt (cacert).
* @param [in] client_key_path
* Path to client.key (cert).
* @param [in] client_crt_path
* Path to client.crt (private-key).
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t ConnectSecured(const char *ca_crt_path, const char *client_key_path, const char *client_crt_path);
/**
* @brief Get the Channel Stub object for communicating with the dpl_rtd.
*
* @return const p4::v1::P4Runtime::Stub* :
* Valid pointer - in case of success.
* NULL - in case of failure.
*/
DOCA_EXPERIMENTAL
const p4::v1::P4Runtime::Stub *GetChannelStub();
/**
* @brief Get the ClientReaderWriter object for communicating with the dpl_rtd.
*
* @return const p4::v1::P4Runtime::Stub* :
* Valid pointer - in case of success.
* NULL - in case of failure.
*/
DOCA_EXPERIMENTAL
const grpc::ClientReaderWriter<p4::v1::StreamMessageRequest, p4::v1::StreamMessageResponse> *GetClientReaderWriter();
/*
* -----------------------------------------------------------------------
* Utility methods
* -----------------------------------------------------------------------
*/
/**
* @brief Load a program on the dpl_rtd.
*
* @param [in] p4info_path
* Path to the compiled p4info.txt file of the program.
* @param [in] blob_path
* Path to the compiled dplconfig blob file of the program.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t LoadProgram(const char *p4info_path, const char *blob_path);
/**
* @brief Get the p4::config::v1::P4Info of the loaded program.
*
* @return const p4::config::v1::P4Info* :
* Valid pointer - in case of success.
* NULL - in case of failure.
*/
DOCA_EXPERIMENTAL
const p4::config::v1::P4Info *GetP4Info();
/*
* -----------------------------------------------------------------------
* Infrastructure methods
* -----------------------------------------------------------------------
*/
/**
* @brief Get gRPC operations callbacks for working on regular tables from the generic dpl_rt_controller APIs.
*
* @return struct dpl_rt_controller_grpc_ops* :
* Valid pointer - in case of success.
* NULL - in case of failure.
*/
DOCA_EXPERIMENTAL
const struct dpl_rt_controller_grpc_ops *GetGrpcOps(void);
/**
* @brief Get the defined Election IDs.
*/
DOCA_EXPERIMENTAL
void GetElectionId(uint64_t *high, uint64_t *low);
/**
* @brief Get the defined device_id.
*
* @param [out] device_id
*/
DOCA_EXPERIMENTAL
uint32_t GetDeviceId(void);
private:
uint32_t device_id_;
const char *rtd_ipaddr_;
uint64_t election_id_high_;
uint64_t election_id_low_;
std::shared_ptr<grpc::Channel> channel_;
std::unique_ptr<p4::v1::P4Runtime::Stub> stub_;
p4::config::v1::P4Info p4info_;
std::unique_ptr<grpc::ClientReaderWriter<p4::v1::StreamMessageRequest, p4::v1::StreamMessageResponse>> stream_;
grpc::ClientContext stream_context_;
struct dpl_rt_controller_grpc_ops grpc_ops_;
doca_error_t Connect(std::shared_ptr<grpc::ChannelCredentials> creds);
};
/* -----------------------------------------------------------------------
* Helpers for working with Bytestrings.
* ----------------------------------------------------------------------- */
/**
* @brief Template for getting a string object representing converted value to big-endian (i.e. network) byte-order.
*
* This must be used when setting match key and action parameter values on gRPC table entries.
*/
template <typename T>
std::string GetBeByteString(T i, size_t bits)
{
// Ensure T is an integral type
static_assert(std::is_integral<T>::value, "Template parameter must be an integral type");
if constexpr (sizeof(T) == sizeof(uint16_t)) {
i = htons(i);
} else if constexpr (sizeof(T) == sizeof(uint32_t)) {
i = htonl(i);
} else if constexpr (sizeof(T) == sizeof(uint64_t)) {
if (*reinterpret_cast<const char *>("\0\x01") == 0) { // check for little endian
const uint32_t high_part = htonl(static_cast<uint32_t>(i >> 32));
const uint32_t low_part = htonl(static_cast<uint32_t>(i & 0xFFFFFFFFLL));
i = (static_cast<uint64_t>(low_part) << 32) | high_part;
i = i >> (64 - bits);
}
} // else: 8-bits or byte-array assumed already to be in big-endian (i.e. network) byte-order
size_t bytes = (bits + 7) / 8;
return std::string(reinterpret_cast<char *>(&i), bytes);
};
/**
* @brief Template for getting a string object representing converted value to big-endian (i.e. network) byte-order.
*
* This must be used when setting match key and action parameter values on gRPC table entries.
*/
template <typename T>
std::string GetBeByteString(T i)
{
return GetBeByteString(i, sizeof(i) * 8);
}
/**
* @brief Template for getting a string object from a big-endian array.
*
* @note Array is assumed to be in big-endian (i.e. network) byte-order, no byte-order conversion is performed.
*
* This must be used when setting match key and action parameter values on gRPC table entries.
*/
template <typename T, size_t N>
std::string GetByteString(T (&addr)[N])
{
return std::string(reinterpret_cast<const char *>(addr), N * sizeof(T));
}
/**
* @brief Get Hex string representation of a byte string value.
*
* @param [in] value
* Value to translate.
* @return std::string : Hex string representation
*/
std::string ByteStringToHexString(const std::string &value);
}; // namespace DPL_P4RT_Controller
APIs:
/* Opaque structures. */
struct dpl_rt_controller_device;
struct dpl_rt_controller_entry;
/** Possible event types. */
enum dpl_rt_controller_event_type {
/** A new program with High Update Rate tables was loaded. */
DPL_EVENT_PROGRAM_LOADED,
/** Program was unloaded; need to cleanup and reconnect once new program is loaded. */
DPL_EVENT_PROGRAM_UNLOADED,
/** Entry addition failed. */
DPL_EVENT_ERROR_ENTRY_ADD,
/** The dpl_rtd crashed; the controller must be restarted. */
DPL_EVENT_ERROR_DETACHED,
};
/**
* @brief Maximum length of a status messages.
*/
#define DPL_RT_MAX_STATUS_MESSAGE_LEN 128
struct dpl_rt_controller_event_data {
/** Optional details message. */
char message[DPL_RT_MAX_STATUS_MESSAGE_LEN];
/** Event data for applicable types. */
union {
struct {
/** Entry that failed to be added. */
struct dpl_rt_controller_entry *entry;
} entry_add;
};
};
/** Event type and data. */
struct dpl_rt_controller_event {
/** The device that raised the event. */
uint32_t device_id;
/** Event type. */
enum dpl_rt_controller_event_type type;
/** Event data. */
struct dpl_rt_controller_event_data data;
};
/** Event handling callback prototype. */
typedef void (*dpl_rt_controller_event_cb_t)(struct dpl_rt_controller_event *event);
/**
* @brief Attributes for attaching to dpl_rtd shared memory.
*/
struct dpl_rt_controller_context_attr {
/** Events handling callback, to be called when events arise. */
dpl_rt_controller_event_cb_t event_cb;
};
/**
* @brief Attributes for connecting to a device.
*/
struct dpl_rt_controller_device_attr {
/** The Device ID to connect to in the dpl_rtd. */
uint32_t device_id;
/** Program hash from the DPL auto-generated header.
* Required to manage High Update Rate, set to <PROGRAM_NAME>_HASH defined by the DPL compiler generated header.
* Required when without_shm_support=false.
*/
const char *program_hash;
/** When set to true, do not connect over SHM, create a device that supports only regular P4Runtime tables,
* without the ability to manage High Update Rate tables.
* Used to allow using the generic entry management APIs when Update Rate tables not required. */
bool without_shm_support;
};
/**
* @brief Counter data.
*/
struct dpl_rt_controller_counter_data {
/** Counter packets count value. */
uint64_t packet_count;
/** Counter bytes count value. */
uint64_t byte_count;
};
/**
* @brief Opaque type for passing p4::v1::TableEntry pointer.
*/
typedef void *p4_v1_table_entry_ptr_t;
/**
* @brief gRPC function callbacks required to allow managing gRPC tables using the dpl_rt_controller APIs.
*/
struct dpl_rt_controller_grpc_ops {
/** Opaque user data pointer, e.g for passing DPL_P4RT_Controller::Controller pointer. */
void *user_data;
/** Callback for allocating p4::v1::TableEntry. */
doca_error_t (*grpc_entry_alloc)(void *user_data, uint32_t table_id, p4_v1_table_entry_ptr_t *entry);
/** Callback for freeing p4::v1::TableEntry handler */
void (*grpc_entry_free)(void *user_data, p4_v1_table_entry_ptr_t entry);
/** Callback for adding p4::v1::TableEntry to loaded program */
doca_error_t (*grpc_entry_add)(void *user_data, p4_v1_table_entry_ptr_t entry);
/** Callback for deleting p4::v1::TableEntry from loaded program */
doca_error_t (*grpc_entry_delete)(void *user_data, p4_v1_table_entry_ptr_t entry);
/** Callback for reading DirectCounter of p4::v1::TableEntry */
doca_error_t (*grpc_entry_counter)(void *user_data,
p4_v1_table_entry_ptr_t entry,
uint64_t *byte_count,
uint64_t *packet_count);
/** Callback for reading DirectCounter of table's default entry */
doca_error_t (*grpc_default_entry_counter)(void *user_data,
uint32_t table_id,
uint64_t *byte_count,
uint64_t *packet_count);
};
/**
* @brief Table statistics collected by the dpl_rtd and the controller.
*/
struct dpl_rt_controller_table_statistics {
// -- Maintained by the dpl_rtd side:
uint64_t insertions; // The total number of successful entries added to the table.
uint64_t insertion_errors; // The total number of failed entries additions.
uint64_t deletions; // The total number of entries successfully removed from the table.
uint64_t deletion_errors; // The total number of failed entries deletions.
uint64_t noops; // Del entry requested before it was added (both ADD and DEL flags set while entry in INIT).
uint64_t pending_add_dels; // How many entry delete received while entry was still PENDING_ADD.
// -- Maintained by controller library side:
uint64_t entry_add_errors; // When the entry_add function returns a failure.
uint64_t entry_del_errors; // When the entry_del function returns a failure.
};
/* -----------------------------------------------------------------------
* Context management methods.
* ----------------------------------------------------------------------- */
/**
* @brief Attach to dpl_rtd SHared Memory (SHM).
*
* @note This must be called first thing, once per process.
* In case that th dpl_rtd crashes, the controller process must be restarted.
*
* @param [in] attr
* Attributes for attaching.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* DOCA_ERROR_UNSUPPORTED_VERSION - SHM API mismatch; the dpl_rtd and the controller library were compiled using
* different shared memory API versions.
* DOCA_ERROR_ALREADY_EXIST - in case that attach is called more than once.
* DOCA_ERROR_INITIALIZATION - in case of initialization errors.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_attach(struct dpl_rt_controller_context_attr *attr);
/**
* @brief Detach from dpl_rtd shared memory.
*
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_detach(void);
/**
* @brief Creates a device context
*
* There two modes when creating a device:
* 1. attr.without_shm_support = false:
* - Connects to a device over shared memory, to support managing High Update Rate tables.
* - Requires calling dpl_rt_controller_attach() first.
* - Optional support for regular P4Runtime tables can be added by calling dpl_rt_controller_grpc_ops_set().
* 2. attr.without_shm_support = true:
* - Used when user only wants use the generic APIs to manage regular P4Runtime tables.
* - Does not connect to a device over shared memory; No support for managing High Update Rate tables.
* - Requires calling dpl_rt_controller_grpc_ops_set().
* - No need to call dpl_rt_controller_attach() at all.
*
* @note Only one client can be connected to a device over shared memory.
*
* @param [in] attr
* Attributes for connecting.
* @param [out] device
* Pointer to created device handle.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* DOCA_ERROR_AGAIN - the device is not loaded with a program yet.
* DOCA_ERROR_NOT_SUPPORTED - the device is loaded with a program that does not contain High Update Rate tables.
* DOCA_ERROR_UNSUPPORTED_VERSION - program_hash mismatch; the device is loaded with a different program version.
* DOCA_ERROR_IN_USE - another client is already connected to this device.
* Error code - in case of other failures.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_connect(struct dpl_rt_controller_device_attr *attr,
struct dpl_rt_controller_device **device);
/**
* @brief Cleanup and disconnect a device
*
* @note After disconnecting, the dpl_rtd will delete remaining High Update Rate table entries!
*
* @param [in] ctx
* Device to disconnect.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_disconnect(struct dpl_rt_controller_device *device);
/* -----------------------------------------------------------------------
* Entry allocation methods.
* ----------------------------------------------------------------------- */
/**
* @brief Allocate a High Update Rate table entry handle from shared memory.
*
* @note Supported only for tables defined with attribute nv_high_update_rate=true in the program.
*
* @param [in] device
* Device to allocate from.
* @param [in] table_id
* High Update Rate table ID to allocate entry from.
* @param [out] entry
* Pointer for providing the dpl_rt_controller_entry handler.
* @param [out] opaque_entry
* Opaque pointer to the C table entry structure backing this entry.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_table_entry_alloc_shm(struct dpl_rt_controller_device *device,
uint32_t table_id,
struct dpl_rt_controller_entry **entry,
void **opaque_entry);
/**
* @brief Allocate a regular P4Runtime table entry handle using defined gRPC callbacks.
*
* @note Not supported for tables defined with attribute nv_high_update_rate=true in the program.
* @note dpl_rt_controller_grpc_ops_set() must be called prior using this API.
*
* @param [in] device
* Device to allocate from.
* @param [in] table_id
* Regular P4Runtime table ID to allocate entry from.
* @param [out] entry
* @param [out] opaque_entry
* opaque pointer to p4::v1::TableEntry backing this entry.
* Pointer for providing the dpl_rt_controller_entry handler.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_table_entry_alloc_grpc(struct dpl_rt_controller_device *device,
uint32_t table_id,
struct dpl_rt_controller_entry **entry,
p4_v1_table_entry_ptr_t *opaque_entry);
/**
* @brief Free backing memory of entry handler.
*
* @note This is used only in error flows, e.g. after failure to add an entry.
*
* @param [in] entry
* Entry to free.
*/
DOCA_EXPERIMENTAL
void dpl_rt_controller_table_entry_free(struct dpl_rt_controller_entry *entry);
/* -----------------------------------------------------------------------
* Entry management methods.
* ----------------------------------------------------------------------- */
/**
* @brief Add a table entry.
*
* The API is asynchronous non-blocking when adding High Update Rate table entries.
* Returning success means that the entry was sent to the dpl_rtd for insertion.
* In case of a successful insertion, there is no further feedback from the dpl_rtd.
* In case of a failed insertion, an event of type DPL_EVENT_ERROR_ENTRY_ADD is raised.
* It is possible to verify the entry insertion status by checking the table statistics using the
* dpl_rt_controller_table_statistics() API.
*
* The API is synchronous blocking when adding a regular table entries, as it is done using P4Runtime RPC message.
* The returned status reflects whether the dpl_rtd inserted the entry successfully or not.
*
* @note Inserting duplicate entries to High Update Rate tables is not rejected by the dpl_rtd, it is the user
* controller app responsibility to avoid inserting duplicate entries.
* @note There is no API for entry lookup, it is the user controller app responsibility to maintain a map between the
* rule and its dpl_rt_controller_entry handle.
*
* @param [in] entry
* Entry to add.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* DOCA_ERROR_AGAIN - in case posting the request failed due to full requests ring.
* DOCA_ERROR_IN_PROGRESS - in case that another addition operation is still in progress.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_table_entry_add(struct dpl_rt_controller_entry *entry);
/**
* @brief Delete a table entry.
*
* The API is asynchronous non-blocking when deleting High Update Rate table entries.
* Returning success means that the entry was sent to the dpl_rtd for deletion.
* In case of a successful deletion, there is no further feedback from the dpl_rtd.
* In case of a failed deletion, the relevant statistic counters are incremented (no even is raised).
* It is possible to verify the entry deletion status by checking the table statistics using the
* dpl_rt_controller_table_statistics() API.
*
* The API is synchronous blocking when deleting a regular table entries, as it is done using P4Runtime RPC message.
* The returned status reflects whether the dpl_rtd deleted the entry successfully or not.
*
* @note If a duplicate entries were inserted, once any such entry is deleted, the existing handles will be invalidated.
* As such, a completion error will be triggered by entry deletion of the invalidated handle. The error is
* silently ignored by the dpl_rtd and counted by dpl_rt_controller_table_statistics.deletion_errors counter
* for debug purposes.
*
* @param [in] entry
* Entry to delete.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* DOCA_ERROR_AGAIN - in case posting the request failed due to full requests ring.
* DOCA_ERROR_IN_PROGRESS - in case that another deletion operation is still in progress.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_table_entry_delete(struct dpl_rt_controller_entry *entry);
/**
* @brief Read entry direct counter data.
*
* The client application is responsible for synchronizing the entry_delete and entry_counter calls.
*
* @note Supported only if direct_counter was defined on the parent table in the program.
* @note This API is always blocking.
*
* @param [in] entry
* Entry to read its counter data.
* @param [out] counter
* Returned entry counter data.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* DOCA_ERROR_AGAIN - in case posting the request failed due to full requests ring.
* DOCA_ERROR_IN_PROGRESS - in case that another query is still in progress.
* DOCA_ERROR_NOT_FOUND - in case direct_counter was not defined on the table in the program.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_table_entry_counter(struct dpl_rt_controller_entry *entry,
struct dpl_rt_controller_counter_data *counter);
/**
* @brief Read table's default entry direct counter data.
*
* @note Supported only if direct_counter was defined on the parent table in the program.
* @note This API is always blocking.
*
* @param [in] device
* Device to read its default entry counter data.
* @param [in] table_id
* Table ID to read its default entry counter data.
* Can be either a High Update Rate table ID or a regular P4Runtime table ID.
* @param [out] counter
* Returned entry counter data.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* DOCA_ERROR_AGAIN - in case posting the request failed due to full requests ring.
* DOCA_ERROR_IN_PROGRESS - in case that another query is still in progress.
* DOCA_ERROR_NOT_FOUND - in case direct_counter was not defined on the table in the program.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_table_default_entry_counter(struct dpl_rt_controller_device *device,
uint32_t table_id,
struct dpl_rt_controller_counter_data *counter);
/**
* @brief Get the opaque pointer to the C table entry structure backing this entry.
*
* The returned opaque pointer points to memory buffer for providing the table entry details (keys, params, etc),
* which must be casted to the corresponding table entry C structure defined at the C header generated by the DPL
* Compiler.
*
* @note Supported only for entries allocated using dpl_rt_controller_table_entry_alloc_shm().
*
* @param [in] entry
* dpl_rt_controller_entry
* @return void** :
* Valid pointer for providing the opaque entry structure pointer - in case of success.
* NULL - in case of failure.
*/
DOCA_EXPERIMENTAL
void *dpl_rt_controller_table_entry_get_shm(struct dpl_rt_controller_entry *entry);
/**
* @brief Get the opaque pointer to p4::v1::TableEntry backing this entry.
*
* The returned opaque pointer points to a p4::v1::TableEntry for providing the table entry details (keys, params, etc).
*
* @note Supported only for entries allocated using dpl_rt_controller_table_entry_alloc_grpc().
*
* @param [in] entry
* dpl_rt_controller_entry
* @return void** :
* Valid pointer for providing the p4::v1::TableEntry pointer - in case of success.
* NULL - in case of failure.
*/
DOCA_EXPERIMENTAL
p4_v1_table_entry_ptr_t dpl_rt_controller_table_entry_get_grpc(struct dpl_rt_controller_entry *entry);
/* -----------------------------------------------------------------------
* gRPC integration methods.
* ----------------------------------------------------------------------- */
/**
* @brief Set P4Runtime controller gRPC callbacks for allowing managing regular table entries.
*
* @param [in] device
* Device to set the gRPC callback on.
* @param [in] ops
* The set of user's gRPC callbacks.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_grpc_ops_set(struct dpl_rt_controller_device *device,
const struct dpl_rt_controller_grpc_ops *ops);
/**
* @brief Clear the currently set gRPC callbacks.
*
* @param [in] device
* Device to clear its gRPC callbacks.
*/
DOCA_EXPERIMENTAL
void dpl_rt_controller_grpc_ops_clear(struct dpl_rt_controller_device *device);
/*
* -----------------------------------------------------------------------
* Utility methods
* -----------------------------------------------------------------------
*/
/**
* @brief Get table statistics.
*
* @note Supported only for tables defined with attribute nv_high_update_rate=true in the program.
* @note After a new DPL program is loaded, the 'insertions' counter will have value 1 since a default table entry
* was inserted to the High Update Rate table during loading the program.
*
* @param [in] device
* Device containing the requested table.
* @param [in] table_id
* Table ID to get it's statistics.
* @param [out] stats
* Pointer to struct dpl_rt_controller_table_statistics to return the statistics.
* @return doca_error_t :
* DOCA_SUCCESS - in case of success.
* Error code - in case of failure.
*/
DOCA_EXPERIMENTAL
doca_error_t dpl_rt_controller_table_statistics(struct dpl_rt_controller_device *device,
uint32_t table_id,
struct dpl_rt_controller_table_statistics *stats);
/**
* @brief Get string representation of a dpl_rt_controller_event_type.
*
* @param [in] type
* Event type value to convert.
* @return const char* :
* String representation.
*/
DOCA_EXPERIMENTAL
const char *dpl_rt_controller_event_type_to_str(enum dpl_rt_controller_event_type type);
Sample controller applications for a given DPL programs are installed by the dpl-rt-controller-samples package at the /opt/dpl_rt_controller/samples/ directory.
Running the Sample
Follow these steps to build and run a sample application:
Connect to the DPU.
Make sure the DPL Runtime Service is up and running. See DPL Container Deployment for more details.
Navigate to the desired sample application directory (one of the folders under
/opt/dpl_rt_controller/samples/on the DPU).Compile the provided DPL program (e.g.,
hello_world.p4) found at the sample's directory. See Compiling DPL Applications for more details.Copy the DPL program compilation output directory (
_out) to a local path on the DPU.NoteThe compilation output directory contains the required C header files (
<program name>_dpl_shm.h and <program name>_dpl_shm_id.h) for compiling the sample controller application.Compile the C sample application:
cd/opt/dpl_rt_controller/samples/<sample_name> meson /tmp/build -Dsample_programs_out=<PATH_TO_DPL_PROGRAM_COMPILATION_OUT_FOLDER> ninja -C /tmp/buildNoteReplace the
<PATH_TO_DPL_PROGRAM_COMPILATION_OUT_FOLDER>with the path to the local directory containing the DPL program compilation output folder (where the required C header files (<program name>_dpl_shm.h and <program name>_dpl_shm_id.h) are found).InfoThe binary
dpl_sample_<sample_name>is created under/tmp/build/.Run the sample application using a privileged user. For example:
sudo /tmp/build/dpl_sample_basic 1000 <path to p4info.txt> <path-to-dplconfig>
Samples
Basic
This sample demonstrates how to manage entries on a High Update Rate table as well as a regular table.
The sample logic includes:
Connecting to
dpl_rtdover gRPC.Loading a DPL program.
Connecting to
dpl_rtdover SHM (SHared Memory).Adding entry to a regular table.
Adding entry to a High Update Rate table.
Reading entries counter.
Deleting entries.
Displaying statistics.
Disconnecting and destroying all structures.
References:
/opt/dpl_rt_controller/samples/basic/basic_sample.cc/opt/dpl_rt_controller/samples/basic/basic_main.cc/opt/dpl_rt_controller/samples/basic/hello_world.p4/opt/dpl_rt_controller/samples/basic/meson_options.txt/opt/dpl_rt_controller/samples/basic/meson.build
gRPC Only
This sample demonstrates how to use the generic dpl_rt_controller APIs to manage regular P4Runtime tables when High Update Rate tables are neither required nor defined in the DPL program.
The sample logic includes:
Connecting to
dpl_rtdover gRPC.Loading a DPL program.
Creating
dpl_rt_controllerdevice without SHM support.Adding entry to a regular table.
Reading entries counter.
Deleting entries.
Disconnecting and destroying all structures.
References:
/opt/dpl_rt_controller/samples/grpc_only/grpc_only_sample.cc/opt/dpl_rt_controller/samples/grpc_only/grpc_only_main.cc/opt/dpl_rt_controller/samples/grpc_only/grpc_only.p4/opt/dpl_rt_controller/samples/grpc_only/meson_options.txt/opt/dpl_rt_controller/samples/grpc_only/meson.build