NVIDIA DOCA Secure Channel Application Guide
NVIDIA Docs Hub  NVIDIA Networking  BlueField DPUs / SuperNICs & DOCA  DOCA Documentation v2.6.0  NVIDIA DOCA Secure Channel Application Guide

On This Page

This guide provides a secure channel implementation on top of NVIDIA® BlueField® DPU.

Introduction

The DOCA Secure Channel reference application leverages the DOCA Comm Channel API which creates a secure, network independent communication channel between the host and the NVIDIA BlueField DPU.

Comm channel allows the host to control services on the DPU, activate certain offloads, or exchange messages using client-server framework.

The client (host) side is able to communicate only with one server at a time while the server side is able to communicate with multiple clients.

The API allows communication between any PF/VF/SF on the host to the server located on the DPU.

Secure channel allows the user to select the message size and amount to be exchanged between the client and the server to simulate heavy load on the channel.

System Design

A secure channel application runs on client mode (host) and server mode (DPU). Once a channel is open, messages can flow from both sides.

sys-design-version-1-modificationdate-1707420962150-api-v2.png

Application Architecture

The secure channel application runs on top of the DOCA Comm Channel API. Full connection flow between the client and the server is illustrated in the following:

application-architecture-diagram-version-1-modificationdate-1707420962587-api-v2.png

  1. Both sides initiate create().

  2. Server listens and waits for new connections.

  3. Server initiates recvfrom() to indicate it is ready to exchange messages

  4. Client executes connect() to server and starts connection initialization.

  5. Client sends first message to server.

  6. Server sends a response.

DOCA Libraries

This application leverages the following DOCA library:

Refer to its respective programming guide for more information.

Running the Application

Installation

Refer to the NVIDIA DOCA Installation Guide for Linux for details on how to install BlueField-related software.

Application Execution

The secure channel application is provided in both source and binary forms. The binary is located under /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel.

  1. Application usage instructions:

    Copy
    Copied!
                
    
            
    Usage: doca_secure_channel [DOCA Flags] [Program Flags]
 
DOCA Flags:
  -h, --help                        Print a help synopsis
  -v, --version                     Print program version information
  -l, --log-level                   Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
  --sdk-log-level                   Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
  -j, --json <path>                 Parse all command flags from an input json file
 
Program Flags:
  -s, --msg-size                    Message size to be sent
  -n, --num-msgs                    Number of messages to be sent
  -p, --pci-addr                    DOCA Comm Channel device PCI address
  -r, --rep-pci                     DOCA Comm Channel device representor PCI address (needed only on DPU)

    Note

    This usage printout can be printed to the command line using the -h (or --help) options:

    Copy
    Copied!
                
    
            
    /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -h

    Note

    For additional information, refer to section "Command Line Flags".

  2. CLI example for running the application on the BlueField:

    Copy
    Copied!
                
    
            
    /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -s 256 -n 10 -p 03:00.0 -r 3b:00.0

    Warning

    Both the DOCA Comm Channel device PCIe address (03:00.0) and the DOCA Comm Channel device representor PCIe address (3b:00.0) should match the addresses of the desired PCIe devices.

  3. CLI example for running the application on the host:

    Copy
    Copied!
                
    
            
    /opt/mellanox/doca/applications/secure_channel/bin/doca_secure_channel -s 256 -n 10 -p 3b:00.0

    Warning

    The DOCA Comm Channel device PCIe address (3b:00.0) should match the address of the desired PCIe device.

  4. The application also supports a JSON-based deployment mode, in which all command-line arguments are provided through a JSON file:

    Copy
    Copied!
                
    
            
    doca_secure_channel --json [json_file]

    For example:

    Copy
    Copied!
                
    
            
    cd /opt/mellanox/doca/applications/secure_channel/bin
./doca_secure_channel --json ./sc_params.json

    Warning

    Before execution, ensure that the used JSON file contains the correct configuration parameters, and especially the PCIe addresses necessary for the deployment.

Command Line Flags

Flag Type

Short Flag

Long Flag/JSON Key

Description

JSON Content

General flags

h

help

Prints a help synopsis

N/A

v

version

Prints program version information

N/A

l

log-level

Set the log level for the application:

  • DISABLE=10

  • CRITICAL=20

  • ERROR=30

  • WARNING=40

  • INFO=50

  • DEBUG=60

  • TRACE=70 ( requires compilation with TRACE log level support )
Copy
Copied!
            

            
"log-level": 60

N/A

sdk-log-level

Sets the log level for the program:

  • DISABLE=10

  • CRITICAL=20

  • ERROR=30

  • WARNING=40

  • INFO=50

  • DEBUG=60

  • TRACE=70
Copy
Copied!
            

            
"sdk-log-level": 40

j

json

Parse all command flags from an input JSON file

N/A

Program flags

s

msg-size

Message size in bytes

Warning

This is a mandatory flag.

Copy
Copied!
            

            
"msg-size": 128

n

num-msgs

Number of messages to send on both sides

Warning

This is a mandatory flag.

Copy
Copied!
            

            
"num-msgs": 256

p

pci-addr

DOCA Comm Channel device PCIe address

Warning

This is a mandatory flag.

Copy
Copied!
            

            
"pci-addr": 03:00.1

r

rep-pci

DOCA Comm Channel device representor PCIe address

Warning

This is a mandatory flag only on the DPU.

Copy
Copied!
            

            
"rep-pci": b1:00.1
Note

Refer to DOCA Arg Parser for more information regarding the supported flags and execution modes.


Troubleshooting

Refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the installation or execution of the DOCA applications .

Recompiling the Application

In addition to providing the application in binary form, the installation also includes all of the application sources and compilation instructions so as to allow modifying the sources and recompiling the application. For more information about the applications, as well as development and compilation tips, refer to the DOCA Applications page.

The sources of the application can be found under the /opt/mellanox/doca/applications/secure_channel/srcdirectory.

Recompiling All Applications

The applications are all defined under a single meson project, so the default compilation recompiles all the DOCA applications.

To build all the applications together, run:

Copy
Copied!
            

            
cd /opt/mellanox/doca/applications/
meson /tmp/build 
ninja -C /tmp/build

Note

doca_secure_channel is created under /tmp/build/secure_channel/src/.


Recompiling Only the Current Application

To directly build only the secure channel application:

Copy
Copied!
            

            
cd /opt/mellanox/doca/applications/
meson /tmp/build -Denable_all_applications=false -Denable_secure_channel=true
ninja -C /tmp/build

Warning

doca_secure_channel will be created under /tmp/build/secure_channel/src/.

Alternatively, users can set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:

  1. Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:

    • Set enable_all_applications to false

    • Set enable_secure_channel to true

  2. Run the following compilation commands :

    Copy
    Copied!
                
    
            
    cd /opt/mellanox/doca/applications/
meson /tmp/build 
ninja -C /tmp/build

    Note

    doca_secure_channel is created under /tmp/build/secure_channel/src/.

Troubleshooting

Refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the compilation of the application .

Application Code Flow

  1. Parse application argument.

    1. Initialize the arg parser resources and register DOCA general parameters.

      Copy
      Copied!
                  
      
            
      doca_argp_init();

    2. Register secure channel application parameters.

      Copy
      Copied!
                  
      
            
      register_secure_channel_params();

    3. Parse application parameters:

      Copy
      Copied!
                  
      
            
      doca_argp_start();

  2. Run main logic.

    Copy
    Copied!
                
    
            
    sc_start();

    1. Initiate synchronization mechanism between send and receive threads.

    2. Initiate Comm Channel endpoint.

    3. Server side starts listening for new connections and client side connects to server.

    4. Initiate signal masking and epoll file descriptor.

    5. Start send and receive threads. Both threads share the same Comm Channel so each one must "lock" the channel before any send/receive operation.

    6. Send thread prints total number of messages successfully sent.

    7. Once Ctrl+C is entered in the shell, receive thread prints the total number of messages successfully received.

    8. Close and destroy resources.

References

  • /opt/mellanox/doca/applications/secure_channel/src

  • /opt/mellanox/doca/applications/secure_channel/bin/sc_params.json

© Copyright 2023, NVIDIA. Last updated on Feb 9, 2024.