Use the script /usr/sbin/ofed_uninstall.sh to uninstall the NVIDIA OFED package. The script is part of the ofed-scripts RPM.
Uninstalling NVIDIA OFED Using the YUM and apt-get Tools
NVIDIA MLNX_OFED Documentation v23.10-3.2.1.1 LTS