Kubernetes Deployment#
NIM LLM 2.0 supports deployment on Kubernetes using Helm charts, NIM Operator, and other orchestration platforms. Select your deployment or platform below for detailed setup instructions.
Note
For CSP-managed Kubernetes environments (such as GKE, EKS, and AKS), refer to the CSP Deployment guides.
Deployment Options#
Use one of the following guides to deploy NVIDIA NIM for LLMs on Kubernetes:
Helm and Kubernetes
Deploy NVIDIA NIM for LLMs on Kubernetes by using the NIM Helm chart.
KServe
Deploy NVIDIA NIM for LLMs on KServe in a Kubernetes environment.
OpenShift
Deploy NVIDIA NIM for LLMs on Red Hat OpenShift, including GPU operator setup and verification.
Run:ai
Deploy NVIDIA NIM for LLMs on Run:ai for inference workloads.
NIM Operator Deployment
Deploy NVIDIA NIM for LLMs by using the NVIDIA NIM Operator on Kubernetes.