Enhanced Network Stack (ENS), also appears as Enhanced Data Path is a networking stack mode, which when configured provides superior network performance. It is primarily targeted for NFV workloads, which requires the performance benefits provided by this mode. ENS utilizes DPDK Poll Mode driver model and significantly improves packet rate and latency for small message sizes.
Current driver can operate in both ENS and legacy (slow path) modes. Device mode of operation is determined automatically based on Virtual Switch mode. Once the uplink is attached to NSX-T Virtual Distribute Switch (N-VDS) which is configured for ENS, the driver will be re-attached to the device and will initialize it to work with ENS.
Please follow VMWare documentation for N-VDS configuration instructions:
To achieve best performance, it is recommended to have N-VDS assigned with NUMA node which is local to Mellanox NIC. NUMA node and amount of logical cores for enhanced data- path processing is selected during the initial N-VDS configuration.
For further information, refer to: https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-D7CA778B-6554-4A23-879D- 4BC336E01031.html#GUID-D7CA778B-6554-4A23-879D-4BC336E01031
To find out the number for NUMA nodes on your host, run the “esxcli hardware memory get” command.
To find out the NIC’s affinity, run the following command: vsish -e cat /net/pNics/<vmnicX>/properties | grep -i numa
The following are the current ENS limitations:
The device does not support SR-IOV when attached to ENS N-VDS. In such configuration, max_vfs module parameter for the ENS port will be ignored and no Virtual Functions will be created for this port. Meaning, if we have 2-port devices with vmnic4 and vmnic5 uplinks connected to a regular ENS and ENS DVS respectively, no VFs will be created for vmnic5 PF.
RDMA and ENS are mutually exclusive features in this release. Do not attach the uplink to ENS N-VDS when using PVRDMA or RoCE on this device.
RSS is not supported currently in ENS mode.
VXLAN is not supported currently in ENS mode.