Installation Scenarios#

This section describes common scenarios to install the NeMo microservices.

NVIDIA provides flexible installation options for you to set up the NeMo microservices for various AI and machine learning tasks. This page outlines several scenarios of installing the NeMo microservices for various use cases.

Before Installing NeMo Microservices#

Before installing the NeMo microservices, get familiar with common prerequisites and tasks for using the NeMo Helm charts introduced in the following guide.

Install Entire NeMo Microservices as a Platform#

Install the entire NeMo microservices as a platform using the NeMo Microservices Helm Chart. This is the parent chart that includes all NeMo microservices and their dependencies as subcharts. For more information, refer to the following Helm installation guide.

After installation, your data scientists and AI application developers can fully utilize the NeMo microservices user guides.

Install a Subset of NeMo Microservices#

If you want to install a subset of the NeMo microservices, you have two options:

  • Option 1: Use the NeMo Microservices Helm Chart to install the subset of the NeMo microservices by disabling the subcharts that you don’t need in the values file for the chart. To view the chart default values, refer to NeMo Microservices Helm Chart. You can disable the subchart installation by setting the <microservice-name>.enabled parameters to false in the values file.

  • Option 2: The NeMo microservices support component-based deployment, which is useful if you require them to be in separate namespaces or clusters. Individual component Helm charts are available for you to fetch from the NGC Catalog. If you choose this option, proceed to the following subsections to learn about scenarios of installing the individual microservices Helm charts.

Install Microservices for Model Fine-tuning#

To set up a model fine-tuning service on your Kubernetes cluster, install the NeMo Customizer microservice and its dependencies. For more information, refer to the following guide.

After installation, your data scientists can follow the user guide at About Fine-Tuning.

Install Microservices for Model Fine-tuning and Evaluation Loop#

To set up a model fine-tuning and evaluation loop on your Kubernetes cluster, install the NeMo Customizer and NeMo Evaluator microservices along with their dependencies. For more information, refer to the following guides.

After installation, your data scientists can use the user guides at About Fine-Tuning and About Evaluating.

Install Microservices for Deploying, Managing, and Proxying NIM for LLMs#

To create an AI inference service with NIM for LLMs, install the NeMo Deployment Management and NIM Proxy microservices and their dependencies. These microservices offer tools to deploy, manage, and proxy the deployed NIMs on your Kubernetes cluster. For more information, refer to the following guides.

After installation, your AI app developers can follow the user guide at About Deploying and Proxying NIM for LLMs.

Install Microservices for Adding Safety Checks to NIM for LLMs#

To add safety checks to NIM for LLMs, install the NeMo Guardrails microservice and its dependencies. If you also want to add model evaluation functionality, install the NeMo Evaluator microservice and its dependencies. For more information, refer to the following guides.

After installation, your data scientists and AI app developers can follow the user guide at About Guardrails.

Install NeMo Microservices to Kubernetes Clusters in the Cloud#

To set up the NeMo microservices to run massive AI workloads on cloud such as multi-node distributed training, install the DGX Cloud Admission Controller Helm chart. This chart helps configure the NeMo microservices to use cloud provider’s networking solutions. For more information, refer to the following guide.

You can also set up the NeMo microservices to use cloud provider’s database services. For more information, refer to the following guide.