image image image image image

On This Page

This feature is supported on ConnectX-6 Dx crypto cards only.


Transport Layer Security (TLS) is a widely-deployed protocol used for securing TCP connections on the Internet. TLS is also a required feature for HTTP/2, the latest web standard. Kernel implementation of TLS (kTLS) provides new opportunities for offloading the protocol into the hardware.

TLS data-path offload allows the NIC to accelerate encryption, decryption and authentication of AES-GCM. TLS offload handles data as it goes through the device without storing any data, but only updating context. If the packet cannot be encrypted/decrypted by the device, then a software fallback handles the packet.

Establishing a kTLS Connection

To avoid unnecessary complexity in the kernel, the TLS handshake is kept in the user space. A full TLS connection using the socket is done using the following scheme.

  1. Call connect() or accept() on a standard TCP file descriptor.
  2. Use a user space TLS library to complete a handshake. 
  3. Create a new KTLS socket file descriptor.
  4. Extract the TLS Initialization Vectors (IVs), session keys, and sequence IDs from the TLS library. Use the setsockopt function on the kTLS file descriptor (FD) to pass them to the kernel.
  5. Use standard read()write()sendfile() and splice() system calls on the KTLS FD.

Drivers can offer Tx and Rx packet encryption/decryption offload from the kernel into the NIC hardware. Upon receipt of a non-data TLS message (a control message), the kTLS socket returns an error, and the message is left on the original TCP socket instead. The kTLS socket is automatically unattached. Transfer of control back to the original encrypted FD is done by calling getsockopt to receive the current sequence numbers, and inserting them into the TLS library.

Kernel Support 

  1. Make sure FreeBSD-13-current is installed and its kernel sources are newer than r362272. Run cd /usr/src
  2. Add the needed options to the kernel's configuration file. 

    {echo "options KERN_TLS" >> sys/amd64/conf/GENERIC # Build kernel make buildkernel KERNCONF=GENERIC # Install kernel make installkernel KERNCONF=GENERIC # Reboot reboot # Load the mlx5en(4) driver as usual kldload mlx5en

Configuring kTLS Offloads

To enable kTLS Tx offload, run: 

sysctl kern.ipc.tls.enable=1
sysctl kern.ipc.mb_use_ext_pgs=1
sysctl kern.ipc.tls.ifnet.permitted=1
ifconfig mce<N> txtls

To disable kTLS Tx offload, run:

sysctl kern.ipc.tls.enable=0
sysctl kern.ipc.mb_use_ext_pgs=1
sysctl kern.ipc.tls.ifnet.permitted=0
ifconfig mce<N> -txtls

Userspace Support

To build kTLS support in the userspace:

  1. Install OpenSSL from /usr/ports/security/openssl. 

    cd /usr/ports/security/openssl
  2. Ensure the kTLS option is enabled - this option is not enabled by default. 

    make config
    make all install clean
  3. Install a special version of iperf with SSL support.  

    mkdir /usr/img cd /usr/img git clone
    cd iperf_ssl.git
    env LDFLAGS="-L/usr/local/lib -lcrypto -lssl" CFLAGS="-I/usr/local/include" ./configure make all install
  4. Start iperf SSL enabled server. 

    iperf --tls=v1.2 -s
  5. Start iperf SSL enabled client. 

    iperf --tls=v1.2 -c <ipaddress> -t 60 -i 1
  6. Verify that TLS offloading is active by checking the following statistics. 

    sysctl dev.mce.<N>.tls.stats