Release Notes#

v2.0.0#

Features#

  • Added support for deploying the NVIDIA NeMo microservices as custom resources with the NVIDIA NIM Operator. NVIDIA NeMo microservices are a modular set of tools that you can use to customize, evaluate, and secure large language models (LLMs) while optimizing AI applications across on-premises or cloud-based Kubernetes clusters. Deploying these mircoservices with the NIM Operator provides the ability to manage your AI workflows across your Kubernetes cluster.

    The NIM Operator supports deploying the following NeMo microservices to your cluster as custom resources:

    • NeMo core microservices

      • NeMo Customizer

      • NeMo Evaluator

      • NeMo Guardrails

    • NeMo platform microservices

      • NeMo Data Store

      • NeMo Entity Store

    Get started with the NeMo microservices.

    Refer to the NeMo microservices documentation for details the using thse microservices.

  • Improved NIMService status to detail model information including the cluster endpoint, external endpoint, and name of the model the service is connected to.

  • Updated required fields for NIMService and NIMPipeline custom resources including:

    • nimservice.spec.image.respository

    • nimservice.spec.image.tag

    • nimpipeline.spec.image.respository

    • nimpipeline.spec.image.tag

    • nimpipeline.spec.expose

  • Enable caching of pre-built TRT-LLM engines and optimized artifacts for non-LLM NIMs, such as Riva and BioNeMo NIMs.

  • Add support for configuring annotations and security contexts for the NIM Operator deployment via Helm values, fixing issue #333.

  • Decouple NIM Operator upgrades from NIMService upgrades. NIMService pods are now only restarted when their corresponding CR specifications are updated.

  • Improve support for clusters operating behind HTTP proxies, including injection of proxy environment variables and custom CA certificates.

Bug fixes#

  • Fixed an issue where an empty storage class was set in the PVCs created by the NIM Operator for caching NIMs.

v1.0.1#

Breaking Changes#

  • Renamed the spec.gpuSelectors field in the NIM cache custom resource to spec.nodeSelector. The purpose of the field remains the same–to specify the node selector labels for scheduling the caching job. Refer to About the NIM Cache Custom Resource Definition.

  • Changed the Operator pod metrics from HTTPS protocol on port 8443 to HTTP protocol on port 8080.

Features#

  • Added a spec.env field to the NIM cache custom resource to support environment variables. One use of the field is to specify variables such as HTTPS_PROXY for air-gapped and specialized networks. Refer to Caching Models in Air-Gapped Environments.

  • Updated the spec.expose.service.type field in the NIM service custom resource to support common service types, such as LoadBalancer.

  • Added a spec.runtimeClassName field to the NIM service custom resource to support setting the runtime class on a NIM service deployment.

  • Removed the kube-rbac-proxy container from the Operator pod. This change improves the security posture of the Operator. Previously, you might need to provide TLS certificates when you configured Prometheus. With this release, you no longer need to provide the certificates.

  • Certified the Operator for use with Red Hat OpenShift Container Platform.

v1.0.0#

Features#

  • NVIDIA NIM Operator is new.

Known Issues#

  • The container versions for the NeMo Retriever Text Embedding NIM and NeMo Retriever Text Reranking NIM are not publicly available and result in an image pull back off error. The Operator and documentation were developed with release candidate versions of these microservices.

  • The Operator does not support configuring NIM microservices in a multi-node deployment.

  • For VMware vSphere with Tanzu clusters using vGPU software, to use an inference model that requires more than one GPU, the NVIDIA A100 or H100 GPUs must be connected with NVLink or NVLink Switch. These clusters also do not support multi-GPU models with L40S GPUs and vGPU software.

  • The Operator is not verified in an air-gapped network environment.

  • The sample RAG application cannot be deployed on Red Hat OpenShift Container Platform.

  • The Operator has transitive dependency on go.uber.org/zap v1.26.0. Findings indicate Cross-Site Scripting (XSS) vulnerabilities in the Zap package.