One RDMA device (HCA) can be shared among multiple Pods running in a Kubernetes worker nodes. User defined networks are created using VXLAN or VETH networking devices. RDMA device (HCA) can be shared among multiple Pods running in a Kubernetes worker nodes.
Kubernetes with Shared HCA
NVIDIA MLNX_OFED Documentation v23.10-2.1.3.1 LTS