IPsec Crypto Offload
This feature is supported on crypto-enabled products of NVIDIA® BlueField®-2 DPUs, and NVIDIA® ConnectX®-6 Dx and ConnectX-7 adapters (but not of ConnectX-6).
Newer/future crypto-enabled DPU and adapter product generations should also support the feature, unless explicitly stated in their documentation.
For BlueField-2 and ConnectX-6 Dx devices only: If your target application will utilize bandwidth of 100Gb/s or higher, where a substantial part of the bandwidth will be allocated for IPsec traffic, please refer to the NVIDIA BlueField-2 DPUs Product Release Notes or NVIDIA ConnectX-6 Dx Adapters Product Release Notes to learn about a potential bandwidth limitation. To access the relevant product release notes, please contact your NVIDIA sales representative.
IPsec crypto offload feature, also known as IPsec inline offload or IPsec aware offload feature enables the user to offload IPsec crypto encryption and decryption operations to the hardware.
The hardware implementation only supports AES-GCM encryption scheme.
To enable the feature, support in both kernel and adapter firmware is required.
- To add IPsec crypto offload support in the kernel, set the following flags accordingly: - CONFIG_XFRM_OFFLOAD=y CONFIG_INET_ESP_OFFLOAD=m CONFIG_INET6_ESP_OFFLOAD=m Note- These flags are enabled by default in RedHat 8 and Ubuntu 18.04.0. 
- To check whether IPsec crypto offload is supported in firmware, look for the following string in the dmesg: - mlx5e: IPSec ESP acceleration enabled 
To program the inline offload security associations (SA), add the option offload dev <netdev interface> dir out/in in the ip xfrm state command for transmitting and receiving SA.
- Transmit inline offload SA xfrm command example: - sudo ip xfrm state add src - 192.168.- 1.64/- 24dst- 192.168.- 1.65/- 24proto esp spi- 0x46dc6204reqid- 0x46dc6204mode transport aead- 'rfc4106(gcm(aes))'- 0x60bd6c3eafba371a46411830fd56c53af93883261ed1fb26767820ff493f43ba35b0dcca- 128offload dev p4p1 dir out sel src- 192.168.- 1.64dst- 192.168.- 1.65
- Receive inline offload SA xfrm command example: - sudo ip xfrm state add src - 192.168.- 1.65/- 24dst- 192.168.- 1.64/- 24proto esp spi- 0xaea0846creqid- 0xaea0846cmode transport aead- 'rfc4106(gcm(aes))'- 0x81d5c3167c912c1dd50dab0cb4b6d815b6ace8844304db362215a258cd19deda8f89deda- 128offload dev p4p1 dir in sel src- 192.168.- 1.65dst- 192.168.- 1.64
Example of setting xfrm policies:
- First server: - + sudo ip xfrm state add src - 192.168.- 1.64/- 24dst- 192.168.- 1.65/- 24proto esp spi- 0x28f39549reqid- 0x28f39549mode transport aead- 'rfc4106(gcm(aes))'- 0x492e8ffe718a95a00c1893ea61afc64997f4732848ccfe6ea07db483175cb18de9ae411a- 128offload dev enp4s0 dir out sel src- 192.168.- 1.64dst- 192.168.- 1.65+ sudo ip xfrm state add src- 192.168.- 1.65/- 24dst- 192.168.- 1.64/- 24proto esp spi- 0x622a73b4reqid- 0x622a73b4mode transport aead- 'rfc4106(gcm(aes))'- 0x093bfee2212802d626716815f862da31bcc7d9c44cfe3ab8049e7604b2feb1254869d25b- 128offload dev enp4s0 dir in sel src- 192.168.- 1.65dst- 192.168.- 1.64+ sudo ip xfrm policy add src- 192.168.- 1.64dst- 192.168.- 1.65dir out tmpl src- 192.168.- 1.64/- 24dst- 192.168.- 1.65/- 24proto esp reqid- 0x28f39549mode transport + sudo ip xfrm policy add src- 192.168.- 1.65dst- 192.168.- 1.64dir in tmpl src- 192.168.- 1.65/- 24dst- 192.168.- 1.64/- 24proto esp reqid- 0x622a73b4mode transport + sudo ip xfrm policy add src- 192.168.- 1.65dst- 192.168.- 1.64dir fwd tmpl src- 192.168.- 1.65/- 24dst- 192.168.- 1.64/- 24proto esp reqid- 0x622a73b4mode transport
- Second server: - + ssh -A -t root - @l-csi-0921d /bin/bash + set -e +- '['- 0==- 1- ']'+ sudo ip xfrm state add src- 192.168.- 1.64/- 24dst- 192.168.- 1.65/- 24proto esp spi- 0x28f39549reqid- 0x28f39549mode transport aead- 'rfc4106(gcm(aes))'- 0x492e8ffe718a95a00c1893ea61afc64997f4732848ccfe6ea07db483175cb18de9ae411a- 128offload dev enp4s0 dir in sel src- 192.168.- 1.64dst- 192.168.- 1.65+ sudo ip xfrm state add src- 192.168.- 1.65/- 24dst- 192.168.- 1.64/- 24proto esp spi- 0x622a73b4reqid- 0x622a73b4mode transport aead- 'rfc4106(gcm(aes))'- 0x093bfee2212802d626716815f862da31bcc7d9c44cfe3ab8049e7604b2feb1254869d25b- 128offload dev enp4s0 dir out sel src- 192.168.- 1.65dst- 192.168.- 1.64+ sudo ip xfrm policy add src- 192.168.- 1.65dst- 192.168.- 1.64dir out tmpl src- 192.168.- 1.65/- 24dst- 192.168.- 1.64/- 24proto esp reqid- 0x622a73b4mode transport + sudo ip xfrm policy add src- 192.168.- 1.64dst- 192.168.- 1.65dir in tmpl src- 192.168.- 1.64/- 24dst- 192.168.- 1.65/- 24proto esp reqid- 0x28f39549mode transport + sudo ip xfrm policy add src- 192.168.- 1.64dst- 192.168.- 1.65dir fwd tmpl src- 192.168.- 1.64/- 24dst- 192.168.- 1.65/- 24proto esp reqid- 0x28f39549mode transport + echo- 'IPSec tunnel configured successfully'