Install NeMo Microservices Helm Chart#

To deploy the NeMo microservices as a platform, follow the steps below.

Requirements#

Review Requirements for NeMo Microservices Platform for the hardware and software requirements for installing the NeMo Microservices Helm Chart.

Download and Install NeMo Microservices Helm Chart#

  1. Download the NeMo Microservices Helm Chart:

    helm fetch https://helm.ngc.nvidia.com/nvidia/nemo-microservices/charts/nemo-microservices-helm-chart-25.7.0.tgz \
   --username='$oauthtoken' \
   --password=$NGC_API_KEY

  2. Review the default values in the NeMo Microservices Helm Chart reference.

    To override the default values, create a custom values file. Consider the following while creating your custom values file:

    • Set up two basic secrets for accessing resources from the NGC catalog: a generic secret and an image pull secret. A generic secret is for accessing models from endpoints on the NGC catalog, and an image pull secret is for pulling container images from the NGC catalog. There are two options:

      • Option 1: This is the recommended option as the secret keeps your NGC API key private. Create secrets and specify them as existing secrets.

        • Create a generic secret named ngc-api with your NGC API key following the instructions at Create a Generic Secret. Set existingSecret: ngc-api to use this secret in the values.yaml file.

        • Create an image pull secret (Docker registry secret) named nvcrimagepullsecret following the instructions at Create an Image Pull Secret. Pass nvcrimagepullsecret to existingImagePullSecret and global.imagePullSecrets. The global.imagePullSecrets is already set to nvcrimagepullsecret by default.

      • Option 2: This option is not recommended unless you want to test in a personal development environment. Create secrets by setting up the ngcAPIKey and imagePullSecret values with your NGC API key in the values.yaml file without specifying values for existingSecret and existingImagePullSecret. This generates the secrets with your NGC API key. However, this option exposes your NGC API key in the values.yaml file or when you run helm get values <release>.

    • NeMo Customizer requires specific storage class settings in customizer.modelsStorage.storageClassName and customizer.customizerConfig.training.pvc.storageClass. For more information, see ReadWriteMany Persistent Volumes.

    • NeMo Deployment Management requires a storage class setting in deployment-management.deployments.defaultStorageClass due to limitations in the NIM Operator microservice.

    • If your cluster has an existing Volcano service in a dedicated namespace, disable the Volcano service installation in the NeMo Customizer subchart. For more information, see the Volcano section in NeMo Customizer Microservice Deployment Guide.

    • For clusters with existing Argo Workflows in a dedicated namespace, disable the default Argo Workflows installation in the evaluator subchart. Configure this through the evaluator object in the NeMo Microservices Helm Chart’s values.yaml. Set the Argo server authentication mode to server. For more information, see the Argo Workflows section in Deploy the NeMo Evaluator Microservice.

    • Configure the Kubernetes storage class for persistent storage for NeMo Data Store, NeMo Customizer, and NeMo Operator. If you have access to a read-write-many storage class such as a network file system (NFS), you can change the strategy for NeMo Data Store by updating data-store.strategy.type to Rollout.

    • Use an external, managed PostgresSQL instance for NeMo Data Store, NeMo Evaluator, NeMo Customizer, and NeMo Evaluator.

    • Use an external, managed S3-compatible object storage for NeMo Data Store.

    • Configure the ingress object to route incoming requests appropriately. For more information, see Ingress Setup for Production Environment.

  3. Install the chart using your values.yaml file:

    helm --namespace default install nemo \
   nemo-microservices-helm-chart \
   -f values.yaml

    The installation process takes approximately 10 minutes for image downloads, container startup, and communication establishment. The time it takes might vary depending on the speed of your network connection. Pods may show pending or restarting states during this time.

  4. Verify the pod status:

    kubectl get pods

    Confirm that running pods display 1/1 or 2/2 in their ready state.