Use the script /usr/sbin/ofed_uninstall.sh to uninstall the NVIDIA OFED package. The script is part of the ofed-scripts RPM.
Uninstalling NVIDIA OFED Using the YUM and apt-get Tools
NVIDIA MLNX_OFED Documentation v5.9-0.5.6.0.127