Kernel Transport Layer Security (kTLS) Offloads

Transport Layer Security (TLS) is a widely deployed protocol used for securing TCP connections on the Internet. TLS is also a required feature for HTTP/2, the latest web standard. Kernel implementation of TLS (kTLS) provides new opportunities for offloading the protocol into the hardware.

TLS data-path offload allows the NIC to accelerate encryption, decryption and authentication of AES-GCM. TLS offload handles data as it goes through the device without storing any data, but only updating context. If the packet cannot be encrypted/decrypted by the device, then a software fallback handles the packet.

To avoid unnecessary complexity in the kernel, the TLS handshake is kept in the user space. A full TLS connection using the socket is done using the following scheme.

1. Call connect() or accept() on a standard TCP file descriptor.

2. Use a user space TLS library to complete a handshake.

3. Create a new KTLS socket file descriptor.

4. Extract the TLS Initialization Vectors (IVs), session keys, and sequence IDs from the TLS library. Use the setsockopt function on the kTLS file descriptor (FD) to pass them to the kernel. The parameters passed for receive and transmit direction are identical. Please refer to TCP_RXTLS_ENABLE and TCP_TXTLS_ENABLE for more information.

5. Use standard read(), write(), sendfile() and splice() system calls on the KTLS FD. Note that the TCP connection must have received and transmitted at least one byte before TLS receive and TLS hardware offload may be enabled.

Drivers can offer Tx and Rx packet encryption/decryption offload from the kernel into the NIC hardware. Upon receipt of a non-data TLS message (a control message), the kTLS socket returns an error, and the message is left on the original TCP socket instead. The kTLS socket is automatically unattached. Transfer of control back to the original encrypted FD is done by calling getsockopt to receive the current sequence numbers and inserting them into the TLS library.

1. Make sure FreeBSD-13-current is installed and its kernel sources are newer than r362272. Run cd /usr/src

2. Add the needed options to the kernel's configuration file.

   Copy

   Copied!

               
   
               
   # Enable the KERN_TLS option:
   echo "options KERN_TLS" >> sys/amd64/conf/GENERIC
   # Build kernel:
   make buildkernel KERNCONF=GENERIC
   # Install kernel:
   make installkernel KERNCONF=GENERIC
   # Reboot:
   Reboot
   # Load the mlx5en(4) driver:
   kldload mlx5en
           

To enable kTLS Tx and Rx offload, run:

            

            
sysctl kern.ipc.tls.enable=1
sysctl kern.ipc.mb_use_ext_pgs=1
sysctl kern.ipc.tls.ifnet.permitted=1
ifconfig mce<N> txtls rxtls
        

To disable kTLS Tx and Rx offload, run:

            

            
sysctl kern.ipc.tls.enable=0
sysctl kern.ipc.mb_use_ext_pgs=1
sysctl kern.ipc.tls.ifnet.permitted=0
ifconfig mce<N> -txtls rxtls
        

To build kTLS support in the userspace:

1. Install OpenSSL from /usr/ports/security/openssl.

   Copy

   Copied!

               
   
               
   cd /usr/ports/security/openssl
           

2. Ensure the kTLS option is enabled - this option is not enabled by default.

   Copy

   Copied!

               
   
               
   make config
   make all install clean
           

3. Install a special version of iperf with SSL support.

   Copy

   Copied!

               
   
               
   mkdir /usr/img cd /usr/img git clone https://github.com/Mellanox/iperf_ssl.git
   cd iperf_ssl.git
   env LDFLAGS="-L/usr/local/lib -lcrypto -lssl" CFLAGS="-I/usr/local/include" 
   ./configure 
   make all install
           

4. Start iperf SSL enabled server.

   Copy

   Copied!

               
   
               
   iperf --tls=v1.2 -s
           

5. Start iperf SSL enabled client.

   Copy

   Copied!

               
   
               
   iperf --tls=v1.2 -c <ipaddress> -t 60 -i 1
           

6. Verify that TLS offloading is active by checking the following statistics.