NVIDIA DOCA Secure Channel Application Guide
This guide provides a secure channel implementation on top of NVIDIA® BlueField® DPU.
The DOCA Secure Channel reference application leverages the DOCA Comm Channel API which creates a secure, network independent communication channel between the host and the NVIDIA BlueField DPU.
Comm channel allows the host to control services on the DPU, activate certain offloads, or exchange messages using client-server framework.
The client (host) side is able to communicate only with one server at a time while the server side is able to communicate with multiple clients.
The API allows communication between any PF/VF/SF on the host to the server located on the DPU.
Secure channel allows the user to select the message size and amount to be exchanged between the client and the server to simulate heavy load on the channel.
A secure channel application runs on client mode (host) and server mode (DPU). Once a channel is open, messages can flow from both sides.
![sys-design-version-1-modificationdate-1702941531133-api-v2.png](https://docscontent.nvidia.com/dims4/default/94f8b06/2147483647/strip/true/crop/782x851+0+0/resize/782x851!/quality/90/?url=https%3A%2F%2Fk3-prod-nvidia-docs.s3.us-west-2.amazonaws.com%2Fbrightspot%2Fconfluence%2F0000018d-9047-d4d2-abcd-b94745260000%2Fdoca%2Farchive%2Fdoca-v2.5.0%2Fimages%2Fdownload%2Fattachments%2F2448907896%2Fsys-design-version-1-modificationdate-1702941531133-api-v2.png)
The secure channel application runs on top of the DOCA Comm Channel API. Full connection flow between the client and the server is illustrated in the following:
![application-architecture-diagram-version-1-modificationdate-1702941531967-api-v2.png](https://docscontent.nvidia.com/dims4/default/4263608/2147483647/strip/true/crop/1163x847+0+0/resize/1163x847!/quality/90/?url=https%3A%2F%2Fk3-prod-nvidia-docs.s3.us-west-2.amazonaws.com%2Fbrightspot%2Fconfluence%2F0000018d-9047-d4d2-abcd-b94745260000%2Fdoca%2Farchive%2Fdoca-v2.5.0%2Fimages%2Fdownload%2Fattachments%2F2448907896%2Fapplication-architecture-diagram-version-1-modificationdate-1702941531967-api-v2.png)
Both sides initiate create().
Server listens and waits for new connections.
Server initiates recvfrom() to indicate it is ready to exchange messages
Client executes connect() to server and starts connection initialization.
Client sends first message to server.
Server sends a response.
This application leverages the following DOCA library:
Refer to its respective programming guide for more information.
Installation
Refer to the NVIDIA DOCA Installation Guide for Linux for details on how to install BlueField-related software.
Application Execution
The secure channel application is provided in both source and binary forms. The binary is located under /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel.
Application usage instructions:
Usage: doca_secure_channel [DOCA Flags] [Program Flags] DOCA Flags: -h, --help Print a help synopsis -v, --version Print program version information -l, --log-level Set the (numeric) log level
for
the program <10
=DISABLE,20
=CRITICAL,30
=ERROR,40
=WARNING,50
=INFO,60
=DEBUG,70
=TRACE> --sdk-log-level Set the SDK (numeric) log levelfor
the program <10
=DISABLE,20
=CRITICAL,30
=ERROR,40
=WARNING,50
=INFO,60
=DEBUG,70
=TRACE> -j, --json <path> Parse all command flags from an input json file Program Flags: -s, --msg-size Message size to be sent -n, --num-msgs Number of messages to be sent -p, --pci-addr DOCA Comm Channel device PCI address -r, --rep-pci DOCA Comm Channel device representor PCI address (needed only on DPU)NoteThis usage printout can be printed to the command line using the -h (or --help) options:
/opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -h
NoteFor additional information, refer to section "Command Line Flags".
CLI example for running the application on the BlueField:
/opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -s
256
-n10
-p03
:00.0
-r 3b:00.0
WarningBoth the DOCA Comm Channel device PCIe address (03:00.0) and the DOCA Comm Channel device representor PCIe address (3b:00.0) should match the addresses of the desired PCIe devices.
CLI example for running the application on the host:
/opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -s
256
-n10
-p 3b:00.0
WarningThe DOCA Comm Channel device PCIe address (3b:00.0) should match the address of the desired PCIe device.
The application also supports a JSON-based deployment mode, in which all command-line arguments are provided through a JSON file:
doca_secure_channel --json [json_file]
For example:
cd /opt/mellanox/doca/applications/secure_channel/bin ./doca_secure_channel --json ./sc_params.json
WarningBefore execution, ensure that the used JSON file contains the correct configuration parameters, and especially the PCIe addresses necessary for the deployment.
Command Line Flags
Flag Type |
Short Flag |
Long Flag/JSON Key |
Description |
JSON Content |
General flags |
h |
help |
Prints a help synopsis |
N/A |
v |
version |
Prints program version information |
N/A |
|
l |
log-level |
Set the log level for the application:
|
|
|
N/A |
sdk-log-level |
Sets the log level for the program:
|
|
|
j |
json |
Parse all command flags from an input JSON file |
N/A |
|
Program flags |
s |
msg-size |
Message size in bytes Warning
This is a mandatory flag.
|
|
n |
num-msgs |
Number of messages to send on both sides Warning
This is a mandatory flag.
|
|
|
p |
pci-addr |
DOCA Comm Channel device PCIe address Warning
This is a mandatory flag.
|
|
|
r |
rep-pci |
DOCA Comm Channel device representor PCIe address Warning
This is a mandatory flag only on the DPU.
|
|
Refer to DOCA Arg Parser for more information regarding the supported flags and execution modes.
Troubleshooting
Refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the installation or execution of the DOCA applications .
In addition to providing the application in binary form, the installation also includes all of the application sources and compilation instructions so as to allow modifying the sources and recompiling the application. For more information about the applications, as well as development and compilation tips, refer to the DOCA Applications page.
The sources of the application can be found under the /opt/mellanox/doca/applications/secure_channel/srcdirectory.
Recompiling All Applications
The applications are all defined under a single meson project, so the default compilation recompiles all the DOCA applications.
To build all the applications together, run:
cd /opt/mellanox/doca/applications/
meson /tmp/build
ninja -C /tmp/build
doca_secure_channel is created under /tmp/build/secure_channel/src/.
Recompiling Only the Current Application
To directly build only the secure channel application:
cd /opt/mellanox/doca/applications/
meson /tmp/build -Denable_all_applications=false
-Denable_secure_channel=true
ninja -C /tmp/build
doca_secure_channel will be created under /tmp/build/secure_channel/src/.
Alternatively, users can set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:
Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:
Set enable_all_applications to false
Set enable_secure_channel to true
Run the following compilation commands :
cd /opt/mellanox/doca/applications/ meson /tmp/build ninja -C /tmp/build
Notedoca_secure_channel is created under /tmp/build/secure_channel/src/.
Troubleshooting
Refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the compilation of the application .
Parse application argument.
Initialize the arg parser resources and register DOCA general parameters.
doca_argp_init();
Register secure channel application parameters.
register_secure_channel_params();
Parse application parameters:
doca_argp_start();
Run main logic.
sc_start();
Initiate synchronization mechanism between send and receive threads.
Initiate Comm Channel endpoint.
Server side starts listening for new connections and client side connects to server.
Initiate signal masking and epoll file descriptor.
Start send and receive threads. Both threads share the same Comm Channel so each one must "lock" the channel before any send/receive operation.
Send thread prints total number of messages successfully sent.
Once Ctrl+C is entered in the shell, receive thread prints the total number of messages successfully received.
Close and destroy resources.
/opt/mellanox/doca/applications/secure_channel/src
/opt/mellanox/doca/applications/secure_channel/bin/sc_params.json