Helm Installation Scenarios#

This section describes common scenarios to install the NeMo microservices.

NVIDIA provides flexible installation options for you to set up the NeMo microservices for various AI and machine learning tasks. This page outlines several scenarios of installing the NeMo microservices for various use cases.

Before Installing NeMo Microservices#

Before installing the NeMo microservices, get familiar with common prerequisites and tasks for using the NeMo Microservices Helm Chart introduced in the following guide.

Helm Installation Overview

Install Entire NeMo Microservices as a Platform#

Install the entire NeMo microservices as a platform using the NeMo Microservices Helm Chart. This parent chart includes all NeMo microservices and their dependencies as subcharts. If you need to adjust values, you have freedom to create a custom values file to override or use the default values.yaml file for NeMo Microservices Helm Chart. For more information, refer to Install NeMo Microservices as a Platform.

After installation, your data scientists and AI application developers can fully utilize the NeMo microservices capabilities.

Install a Subset of NeMo Microservices#

By default, the NeMo Microservices Helm Chart installs all of the NeMo microservices.

To install a subset of the NeMo microservices, create a custom values file to override or use the default values.yaml file for NeMo Microservices Helm Chart. Disable the subchart installations that you don’t need by setting the <microservice-name>.enabled parameters to false.

Install Microservices for Model Fine-tuning#

To set up the NeMo microservices only for model fine-tuning on your Kubernetes cluster, install the NeMo Customizer microservice and its dependencies.

In the values file, keep the necessary microservices enabled and disable the microservices you don’t need as follows:

customizer:
  enabled: true
evaluator:
  enabled: false
guardrails:
  enabled: false
deployment-management:
  enabled: false
nim-proxy:
  enabled: false

For more specific values configuration for NeMo Customizer, refer to the following guide.

NeMo Customizer Microservice Deployment Guide.

After installation, your data scientists can follow the user guide at About Fine-Tuning and About Managing Entities.

Install Microservices for Model Fine-tuning and Evaluation Loop#

To set up a model fine-tuning and evaluation loop on your Kubernetes cluster, install the NeMo Customizer and NeMo Evaluator microservices along with their dependencies.

In the values file, keep the necessary microservices enabled and disable the rest as follows:

customizer:
  enabled: true
evaluator:
  enabled: true
guardrails:
  enabled: false
deployment-management:
  enabled: false
nim-proxy:
  enabled: false

For more specific values configuration for NeMo Customizer and NeMo Evaluator, refer to the following guides.

After installation, your data scientists can use the user guides at About Fine-Tuning, About Evaluating, and About Managing Entities.

Install Microservices for Deploying, Managing, and Proxying NIM#

To set up services to route requests to NIM, install the NeMo Deployment Management and NIM Proxy microservices. The NeMo Deployment Management microservice offers tools to deploy and manage NIM microservice deployments, and the NIM Proxy microservice proxies the deployed NIM microservices on your Kubernetes cluster.

In the values file, keep the necessary microservices enabled and disable the rest as follows:

customizer:
  enabled: false
evaluator:
  enabled: false
guardrails:
  enabled: false
deployment-management:
  enabled: true
nim-proxy:
  enabled: true

For more specific values configuration for the NeMo Deployment Management and NIM Proxy microservices, refer to the following guides.

After installation, your AI app developers can follow the user guide at About Deploying and Running Inference on NIM.

Install Microservices for Adding Safety Checks to NIM#

To add safety checks to NIM, install the NeMo Guardrails microservice and its dependencies.

By default, the NeMo Microservices Helm Chart has configurations to set up NeMo Guardrails with NeMo Deployment Management and NIM Proxy while NIM is disabled. To use NeMo Guardrails with the two microservices, set the following values in the values.yaml file:

customizer:
  enabled: false
evaluator:
  enabled: false
guardrails:
  enabled: true
deployment-management:
  enabled: true
nim-proxy:
  enabled: true

If you want to directly use NIM with NeMo Guardrails, you can enable NIM deployment in the values file and disable the rest as follows:

customizer:
  enabled: false
evaluator:
  enabled: false
guardrails:
  enabled: true
deployment-management:
  enabled: false
nim-proxy:
  enabled: false
nim:
  enabled: true
  # ... more NIM-specific values

For more specific values configuration for NeMo Guardrails, refer to the following guide.

NeMo Guardrails Microservice Deployment Guide

After installation, your data scientists and AI app developers can follow the user guide at About Guardrails.

Install NeMo Microservices to Kubernetes Clusters in the Cloud#

To set up the NeMo microservices to run AI workloads such as multi-node distributed training on a cloud provider, install the DGX Cloud Admission Controller Helm chart. This chart helps configure the NeMo microservices to use cloud provider’s networking solutions. For more information, refer to the following guide.

DGX Cloud Admission Controller Installation Guide

You can also set up the NeMo microservices to use your cloud provider’s database services. For more information, refer to the following guide.

Manage Storage