NVIDIA DOCA Secure Channel Application Guide

This guide provides a secure channel implementation on top of NVIDIA® BlueField® DPU.

Introduction

The DOCA Secure Channel reference application leverages the DOCA Comm Channel API which creates a secure, network independent communication channel between the host and the NVIDIA BlueField DPU.

Comm channel allows the host to control services on the DPU, activate certain offloads, or exchange messages using client-server framework.

The client (host) side is able to communicate only with one server at a time while the server side is able to communicate with multiple clients.

The API allows communication between any PF/VF/SF on the host to the server located on the DPU.

Secure channel allows the user to select the message size and amount to be exchanged between the client and the server to simulate heavy load on the channel.

System Design

A secure channel application runs on client mode (host) and server mode (DPU). Once a channel is open, messages can flow from both sides.

sys-design-version-1-modificationdate-1702685230547-api-v2.png

Application Architecture

The secure channel application runs on top of the DOCA Comm Channel API. Full connection flow between the client and the server is illustrated in the following:

Both sides initiate create().
Server listens and waits for new connections.
Server initiates recvfrom() to indicate it is ready to exchange messages
Client executes connect() to server and starts connection initialization.
Client sends first message to server.
Server sends a response.

DOCA Libraries

This application leverages the following DOCA library:

DOCA Comm Channel

Refer to its respective programming guide for more information.

Installation

Refer to the NVIDIA DOCA Installation Guide for Linux for details on how to install BlueField-related software.

Application Execution

The secure channel application is provided in both source and binary forms. The binary is located under /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel.

Application usage instructions:

Copy
Copied!

            
            Usage: doca_secure_channel [DOCA Flags] [Program Flags]
 
DOCA Flags:
  -h, --help                        Print a help synopsis
  -v, --version                     Print program version information
  -l, --log-level                   Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
  --sdk-log-level                   Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
  -j, --json <path>                 Parse all command flags from an input json file
 
Program Flags:
  -s, --msg-size                    Message size to be sent
  -n, --num-msgs                    Number of messages to be sent
  -p, --pci-addr                    DOCA Comm Channel device PCI address
  -r, --rep-pci                     DOCA Comm Channel device representor PCI address (needed only on DPU)

Note

This usage printout can be printed to the command line using the -h (or --help) options:

Copy
Copied!

            
            /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -h

Note

For additional information, refer to section "Command Line Flags".

CLI example for running the application on the BlueField:
Copy

Copied!
```
            
            /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -s 256 -n 10 -p 03:00.0 -r 3b:00.0
        
```
Warning

Both the DOCA Comm Channel device PCIe address (03:00.0) and the DOCA Comm Channel device representor PCIe address (3b:00.0) should match the addresses of the desired PCIe devices.
CLI example for running the application on the host:
Copy

Copied!
```
            
            /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -s 256 -n 10 -p 3b:00.0
        
```
Warning

The DOCA Comm Channel device PCIe address (3b:00.0) should match the address of the desired PCIe device.
The application also supports a JSON-based deployment mode, in which all command-line arguments are provided through a JSON file:
Copy

Copied!
```
            
            doca_secure_channel --json [json_file]
        
```
For example:
Copy

Copied!
```
            
            cd /opt/mellanox/doca/applications/secure_channel/bin
./doca_secure_channel --json ./sc_params.json
        
```
Warning

Before execution, ensure that the used JSON file contains the correct configuration parameters, and especially the PCIe addresses necessary for the deployment.

Command Line Flags

Flag Type	Short Flag	Long Flag/JSON Key	Description	JSON Content
General flags	`h`	`help`	Prints a help synopsis	N/A
	`v`	`version`	Prints program version information	N/A
	`l`	`log-level`	Set the log level for the application: DISABLE=10 CRITICAL=20 ERROR=30 WARNING=40 INFO=50 DEBUG=60 TRACE=70 ( requires compilation with `TRACE` log level support )	Copy Copied! `"log-level": 60`
	N/A	`sdk-log-level`	Sets the log level for the program: DISABLE=10 CRITICAL=20 ERROR=30 WARNING=40 INFO=50 DEBUG=60 TRACE=70	Copy Copied! `"sdk-log-level": 40`
	`j`	`json`	Parse all command flags from an input JSON file	N/A
Program flags	`s`	`msg-size`	Message size in bytes Warning This is a mandatory flag.	Copy Copied! `"msg-size": 128`
	`n`	`num-msgs`	Number of messages to send on both sides Warning This is a mandatory flag.	Copy Copied! `"num-msgs": 256`
	`p`	`pci-addr`	DOCA Comm Channel device PCIe address Warning This is a mandatory flag.	Copy Copied! `"pci-addr": 03:00.1`
	`r`	`rep-pci`	DOCA Comm Channel device representor PCIe address Warning This is a mandatory flag only on the DPU.	Copy Copied! `"rep-pci": b1:00.1`

Note

Refer to DOCA Arg Parser for more information regarding the supported flags and execution modes.

Troubleshooting

Refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the installation or execution of the DOCA applications .

In addition to providing the application in binary form, the installation also includes all of the application sources and compilation instructions so as to allow modifying the sources and recompiling the application. For more information about the applications, as well as development and compilation tips, refer to the DOCA Applications page.

The sources of the application can be found under the /opt/mellanox/doca/applications/secure_channel/srcdirectory.

Recompiling All Applications

The applications are all defined under a single meson project, so the default compilation recompiles all the DOCA applications.

To build all the applications together, run:

Copy
Copied!

            
            cd /opt/mellanox/doca/applications/
meson /tmp/build 
ninja -C /tmp/build

Note

doca_secure_channel is created under /tmp/build/secure_channel/src/.

Recompiling Only the Current Application

To directly build only the secure channel application:

Copy
Copied!

            
            cd /opt/mellanox/doca/applications/
meson /tmp/build -Denable_all_applications=false -Denable_secure_channel=true
ninja -C /tmp/build

Warning

doca_secure_channel will be created under /tmp/build/secure_channel/src/.

Alternatively, users can set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:

Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:
- Set enable_all_applications to false
- Set enable_secure_channel to true

Run the following compilation commands :

Copy
Copied!

            
            cd /opt/mellanox/doca/applications/
meson /tmp/build 
ninja -C /tmp/build

Note

doca_secure_channel is created under /tmp/build/secure_channel/src/.

Troubleshooting

Refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the compilation of the application .

Application Code Flow

Parse application argument.

Initialize the arg parser resources and register DOCA general parameters.

Copy
Copied!

            
            doca_argp_init();

Register secure channel application parameters.

Copy
Copied!

            
            register_secure_channel_params();

Parse application parameters:

Copy
Copied!

            
            doca_argp_start();

Run main logic.
Copy

Copied!
```
            
            sc_start();
        
```
1. Initiate synchronization mechanism between send and receive threads.
2. Initiate Comm Channel endpoint.
3. Server side starts listening for new connections and client side connects to server.
4. Initiate signal masking and epoll file descriptor.
5. Start send and receive threads. Both threads share the same Comm Channel so each one must "lock" the channel before any send/receive operation.
6. Send thread prints total number of messages successfully sent.
7. Once Ctrl+C is entered in the shell, receive thread prints the total number of messages successfully received.
8. Close and destroy resources.

References

/opt/mellanox/doca/applications/secure_channel/src
/opt/mellanox/doca/applications/secure_channel/bin/sc_params.json

On This Page

NVIDIA DOCA Secure Channel Application Guide

Introduction

System Design

Application Architecture

DOCA Libraries

Running the Application

Installation

Application Execution

Command Line Flags

Troubleshooting

Recompiling the Application

Recompiling All Applications

Recompiling Only the Current Application

Troubleshooting

Application Code Flow

References