IPsec Crypto Offload
This feature is supported on crypto-enabled products of NVIDIA® BlueField®-2 DPUs, and NVIDIA® ConnectX®-6 Dx and ConnectX-7 adapters (but not of ConnectX-6).
Newer/future crypto-enabled DPU and adapter product generations should also support the feature, unless explicitly stated in their documentation.
For BlueField-2 and ConnectX-6 Dx devices only: If your target application will utilize bandwidth of 100Gb/s or higher, where a substantial part of the bandwidth will be allocated for IPsec traffic, please refer to the NVIDIA BlueField-2 DPUs Product Release Notes or NVIDIA ConnectX-6 Dx Adapters Product Release Notes to learn about a potential bandwidth limitation. To access the relevant product release notes, please contact your NVIDIA sales representative.
IPsec crypto offload feature, also known as IPsec inline offload or IPsec aware offload feature enables the user to offload IPsec crypto encryption and decryption operations to the hardware.
The hardware implementation only supports AES-GCM encryption scheme.
To enable the feature, support in both kernel and adapter firmware is required.
To add IPsec crypto offload support in the kernel, set the following flags accordingly:
CONFIG_XFRM_OFFLOAD=y CONFIG_INET_ESP_OFFLOAD=m CONFIG_INET6_ESP_OFFLOAD=m
NoteThese flags are enabled by default in RedHat 8 and Ubuntu 18.04.0.
To check whether IPsec crypto offload is supported in firmware, look for the following string in the dmesg:
mlx5e: IPSec ESP acceleration enabled
To program the inline offload security associations (SA), add the option offload dev <netdev interface> dir out/in in the ip xfrm state command for transmitting and receiving SA.
Transmit inline offload SA xfrm command example:
sudo ip xfrm state add src
192.168.1.64/24dst192.168.1.65/24proto esp spi0x46dc6204reqid0x46dc6204mode transport aead'rfc4106(gcm(aes))'0x60bd6c3eafba371a46411830fd56c53af93883261ed1fb26767820ff493f43ba35b0dcca128offload dev p4p1 dir out sel src192.168.1.64dst192.168.1.65
Receive inline offload SA xfrm command example:
sudo ip xfrm state add src
192.168.1.65/24dst192.168.1.64/24proto esp spi0xaea0846creqid0xaea0846cmode transport aead'rfc4106(gcm(aes))'0x81d5c3167c912c1dd50dab0cb4b6d815b6ace8844304db362215a258cd19deda8f89deda128offload dev p4p1 dir in sel src192.168.1.65dst192.168.1.64
Example of setting xfrm policies:
First server:
+ sudo ip xfrm state add src
192.168.1.64/24dst192.168.1.65/24proto esp spi0x28f39549reqid0x28f39549mode transport aead'rfc4106(gcm(aes))'0x492e8ffe718a95a00c1893ea61afc64997f4732848ccfe6ea07db483175cb18de9ae411a128offload dev enp4s0 dir out sel src192.168.1.64dst192.168.1.65+ sudo ip xfrm state add src192.168.1.65/24dst192.168.1.64/24proto esp spi0x622a73b4reqid0x622a73b4mode transport aead'rfc4106(gcm(aes))'0x093bfee2212802d626716815f862da31bcc7d9c44cfe3ab8049e7604b2feb1254869d25b128offload dev enp4s0 dir in sel src192.168.1.65dst192.168.1.64+ sudo ip xfrm policy add src192.168.1.64dst192.168.1.65dir out tmpl src192.168.1.64/24dst192.168.1.65/24proto esp reqid0x28f39549mode transport + sudo ip xfrm policy add src192.168.1.65dst192.168.1.64dir in tmpl src192.168.1.65/24dst192.168.1.64/24proto esp reqid0x622a73b4mode transport + sudo ip xfrm policy add src192.168.1.65dst192.168.1.64dir fwd tmpl src192.168.1.65/24dst192.168.1.64/24proto esp reqid0x622a73b4mode transportSecond server:
+ ssh -A -t root
@l-csi-0921d /bin/bash + set -e +'['0==1']'+ sudo ip xfrm state add src192.168.1.64/24dst192.168.1.65/24proto esp spi0x28f39549reqid0x28f39549mode transport aead'rfc4106(gcm(aes))'0x492e8ffe718a95a00c1893ea61afc64997f4732848ccfe6ea07db483175cb18de9ae411a128offload dev enp4s0 dir in sel src192.168.1.64dst192.168.1.65+ sudo ip xfrm state add src192.168.1.65/24dst192.168.1.64/24proto esp spi0x622a73b4reqid0x622a73b4mode transport aead'rfc4106(gcm(aes))'0x093bfee2212802d626716815f862da31bcc7d9c44cfe3ab8049e7604b2feb1254869d25b128offload dev enp4s0 dir out sel src192.168.1.65dst192.168.1.64+ sudo ip xfrm policy add src192.168.1.65dst192.168.1.64dir out tmpl src192.168.1.65/24dst192.168.1.64/24proto esp reqid0x622a73b4mode transport + sudo ip xfrm policy add src192.168.1.64dst192.168.1.65dir in tmpl src192.168.1.64/24dst192.168.1.65/24proto esp reqid0x28f39549mode transport + sudo ip xfrm policy add src192.168.1.64dst192.168.1.65dir fwd tmpl src192.168.1.64/24dst192.168.1.65/24proto esp reqid0x28f39549mode transport + echo'IPSec tunnel configured successfully'