About Admin Setup#
This section describes how to set up the NeMo microservices on your Kubernetes cluster using the NeMo Microservices Helm Chart. With this chart, you can deploy the NeMo microservices as a platform or a subset of the microservices as you need.
This Admin Setup chapter is for the following personas.
Cloud administrators: Manage Kubernetes clusters and compute/storage resources. Deploy NeMo microservices to the Kubernetes clusters on premises or cloud.
NeMo Microservices Helm Chart#
The NeMo Microservices Helm Chart is an all-in-one Helm chart that bundles the complete NeMo microservices ecosystem and all required dependencies for full platform deployment.
You can also customize the configuration of your installation by updating the values.yaml file. You can also use the pre-configured tags to install only specific microservices that you need.
For the chart assets and additional details, refer to the NeMo Microservices Collection page in the NVIDIA NGC Catalog.
Deploy the NeMo Microservices as a Platform#
The following sections provide detailed instructions on how to deploy the NeMo microservices platform using the chart.
Install the NeMo microservices platform using the chart on your Kubernetes cluster.
Deploy Functional NeMo Microservices as a Centralized Service#
Use the following guides to deploy the functional microservices to your Kubernetes cluster with the minimal dependencies. These guides also provide more detailed configuration instructions for each microservice.
Deploy the functional microservices by using the tag-based installation method.
Deploy the NeMo Auditor microservice and its dependencies.
Deploy the NeMo Customizer microservice and its dependencies.
Deploy the NeMo Data Designer microservice and its dependencies.
Deploy the NVIDIA NeMo Evaluator microservice and its dependencies.
Deploy the NeMo Guardrails microservice and its dependencies.
Deploy the NeMo Safe Synthesizer microservice and its dependencies.
Other Deployment Options#
You can also deploy the NeMo microservices as a platform using NVIDIA NIM Operator.
Deploy the NeMo microservices as a platform using NVIDIA NIM Operator.
Infrastructure Management#
The following sections provide detailed instructions on how to manage the infrastructure for the NeMo microservices.
Infrastructure Configurations#
Configure infrastructure microservices that set up infrastructure around the functional microservices.
Storage Configuration#
Configure storage for the NeMo microservices.
Configure PostgreSQL databases for the NeMo microservices.
Configure the object storage used for dataset and model files.
Configure the Persistent Volume Claims used by the NeMo Data Store and NeMo Customizer microservices.
Security and Secrets#
Create, access, and update secrets to use across the NeMo microservices.
Manage secrets for pulling container and model images from the NGC registry.
Manage external database secrets used across microservices.
Manage MLflow customizer secrets used across microservices.
Manage Weights & Biases keys used to track and visualize training metrics during fine-tuning.
Other Configurations#
Review and manage other cluster settings.
Review considerations for single and multi-tenancy configurations.
Review and configure how NeMo Microservices use Open Telemetry for observability.
Review Custom Resource Definitions (CRDs) for training jobs, workloads, and entity handling that NeMo Operator manages.