NVIDIA DOCA File Compression Application Guide

This document provides a file compression implementation on top of the NVIDIA® BlueField® DPU.

The file compression application exhibits how to use the DOCA Compress API to compress and decompress data using hardware acceleration as well as sending and receiving it using the DOCA Comch API.

The application's logic includes both a client and a server:

• Client side – the application opens a file, compresses it, and sends the checksum of the source file with the compressed data to the server

• Server side – the application saves the received file in a buffer, decompresses it, and compares the received checksum with the calculated one

The file compression application client runs on the host and the server runs on the DPU.

The file compression application runs on top of the DOCA Comm Channel API to send and receive the file from the host and to the DPU.

1. Connection is established on both sides by DOCA Comm Channel API.

2. Client compresses the data:

   • When compress engine is available – submits compress job with DOCA Compress API and sends the result to the server

   • When compress engine is unavailable – compresses the data in software

3. Client sends the number of messages needed to send the compressed content of the file.

4. Client sends data segments in size of up to 4080 bytes.

5. Server saves the received data in a buffer and submits a decompress job.

6. Server sends an ACK message to the client when all parts of the file are received successfully.

7. Server compares the received checksum to the calculated checksum.

8. Server writes the decompressed data to an output file.

This application leverages the following DOCA libraries:

• DOCA Compress

• DOCA Comch

Refer to their respective programming guide for more information.

The installation of DOCA's reference applications contains the sources of the applications, alongside the matching compilation instructions. This allows for compiling the applications "as-is" and provides the ability to modify the sources, then compile a new version of the application.

The sources of the application can be found under the application's directory: /opt/mellanox/doca/applications/file_compression/.

Compiling All Applications

All DOCA applications are defined under a single meson project. So, by default, the compilation includes all of them.

To build all the applications together, run:

            

            
cd /opt/mellanox/doca/applications/
meson /tmp/build 
ninja -C /tmp/build
        

Compiling File Compression Application Only

To directly build only the f ile compression application:

            

            
cd /opt/mellanox/doca/applications/
meson /tmp/build -Denable_all_applications=false -Denable_file_compression=true
ninja -C /tmp/build
        

Alternatively, the user may set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:

1. Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:

   • Set enable_all_applications to false

   • Set enable_file_compression to true

2. Run the following compilation commands :

   Copy

   Copied!

               
   
               
   cd /opt/mellanox/doca/applications/
   meson /tmp/build 
   ninja -C /tmp/build
           

   Info

   doca_file_compression is created under /tmp/build/file_compression/.

Troubleshooting

Refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the compilation of the application.

Application Execution

The file compression application is provided in source form. Therefore, a compilation is required before the application can be executed.

1. Application usage instructions:

   Copy

   Copied!

               
   
               
   Usage: doca_file_compression [DOCA Flags] [Program Flags]
    
   DOCA Flags:
     -h, --help                        Print a help synopsis
     -v, --version                     Print program version information
     -l, --log-level                   Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
     --sdk-log-level                   Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
     -j, --json <path>                 Parse all command flags from an input json file
    
   Program Flags:
     -p, --pci-addr                    DOCA Comm Channel device PCI address
     -r, --rep-pci                     DOCA Comm Channel device representor PCI address
     -f, --file                        File to send by the client / File to write by the server
     -t, --timeout                     Application timeout for receiving file content messages, default is 5 sec
           

   Info

   This usage printout can be printed to the command line using th e -h (or --help) options:

   Copy

   Copied!

               
   
               
   ./doca_file_compression -h
           

   Info

   For additional information, refer to section "Command Line Flags".

2. CLI example for running the application on BlueField:

   Copy

   Copied!

               
   
               
   ./doca_file_compression -p 03:00.0 -r 3b:00.0 -f received.txt
           

   Note

   Both the DOCA Comm Channel device PCIe address (03:00.0) and the DOCA Comm Channel device representor PCIe address (3b:00.0) should match the addresses of the desired PCIe devices.

3. CLI example for running the application on the host:

   Copy

   Copied!

               
   
               
   ./doca_file_compression -p 3b:00.0 -f send.txt
           

   Note

   The DOCA Comm Channel device PCIe address (3b:00.0) should match the address of the desired PCIe device.

4. The application also supports a JSON-based deployment mode, in which all command-line arguments are provided through a JSON file:

   Copy

   Copied!

               
   
               
   ./doca_file_compression --json [json_file]
           

   For example:

   Copy

   Copied!

               
   
               
   ./doca_file_compression --json ./file_compression_params.json
           

   Note

   Before execution, ensure that the used JSON file contains the correct configuration parameters, and especially the PCIe addresses necessary for the deployment.

Command Line Flags

            

            
"log-level": 60
        

            

            
"sdk-log-level": 40
        

            

            
"file": "/tmp/data.txt"
        

            

            
"pci-addr": 03:00.1 
        

            

            
"rep-pci": b1:00.1 
        

Troubleshooting

Refer to the NVIDIA DOCA Troubleshooting Guide for any issue encountered with the installation or execution of the DOCA applications.

1. Parse application argument.

   1. Initialize arg parser resources and register DOCA general parameters.

      Copy

      Copied!

                  
      
                  
      doca_argp_init();
              

   2. Register file compression application parameters.

      Copy

      Copied!

                  
      
                  
      register_file_compression_params();
              

   3. Parse the arguments.

      Copy

      Copied!

                  
      
                  
      doca_argp_start();
              

      1. Parse app parameters.

2. Set endpoint attributes.

   Copy

   Copied!

               
   
               
   set_endpoint_properties();
           

   1. Set maximum message size of 4080 bytes.

   2. Set maximum number of messages allowed.

3. Create comm channel endpoint.

   Copy

   Copied!

               
   
               
   doca_comm_channel_ep_create();
           

   1. Create endpoint for client/server.

4. Run client/server main logic.

   Copy

   Copied!

               
   
               
   file_compression_client/server();
           

5. Clean up the file compression application.

   Copy

   Copied!

               
   
               
   file_compression_cleanup();
           

   1. Free all application resources.

6. Arg parser destroy.

   Copy

   Copied!

               
   
               
   doca_argp_destroy()
           

• /opt/mellanox/doca/applications/file_compression/

• /opt/mellanox/doca/applications/file_compression/file_compression_params.json