Use the script /usr/sbin/ofed_uninstall.sh to uninstall the NVIDIA OFED package. The script is part of the ofed-scripts RPM.
Uninstalling NVIDIA OFED Using the YUM and apt-get Tools
NVIDIA MLNX_OFED Documentation v5.8-5.1.1.2 LTS