Uninstalling the GPU Operator
Perform the following steps to uninstall the Operator.
Optional: List and delete NVIDIA driver custom resources.
$ kubectl get nvidiadrivers
Example Output
NAME STATUS AGE demo-gold ready 2023-10-16T17:57:12Z demo-silver ready 2023-10-16T17:57:12Z
$ kubectl delete nvidiadriver demo-gold $ kubectl delete nvidiadriver demo-silver
$ kubectl delete crd nvidiadrivers.nvidia.com
Delete the Operator:
$ helm delete -n gpu-operator $(helm list -n gpu-operator | grep gpu-operator | awk '{print $1}')
Optional: List the pods in the Operator namespace to confirm the pods are deleted or in the process of deleting:
$ kubectl get pods -n gpu-operator
Example Output
No resources found.
By default, Helm does not support deleting existing CRDs
when you delete the chart.
As a result, the clusterpolicy
CRD and nvidiadrivers
CRD will still remain, by default.
$ kubectl get crd clusterpolicies.nvidia.com
To overcome this, the Operator uses a post-delete hook
to perform the CRD cleanup.
The operator.cleanupCRD
chart parameter is added to enable this hook.
This parameter is disabled by default.
You can enable the hook by specifying --set operator.cleanupCRD=true
during install or upgrade to perform automatic CRD cleanup on chart deletion.
Alternatively, you can delete the custom resource definition:
$ kubectl delete crd clusterpolicies.nvidia.com
Note
After uninstalling the Operator, the NVIDIA driver modules might still be loaded. Either reboot the node or unload them using the following command:
$ sudo rmmod nvidia_modeset nvidia_uvm nvidia
Helm hooks used with the GPU Operator use the Operator image itself. If the Operator image cannot be pulled successfully (either due to network error or an invalid NGC registry secret in case of NVAIE), hooks will fail. In this case, delete the chart and specify the
--no-hooks
argument to avoid hanging on hook failures.