About Admin Setup#
NeMo microservices offer flexibility for installing entirely as a platform or a portion of the microservices as you need. You can complete the data flywheel with the NeMo microservices on your Kubernetes clusters that run NVIDIA GPUs.
This Admin Setup chapter is for the following persona.
Cloud administrators: Manage Kubernetes clusters and compute/storage resources. Deploy NeMo platform components to the Kubernetes clusters on premises or cloud.
NeMo Helm Charts#
The Helm charts for NeMo microservices are in the following two categories.
NeMo Microservices Helm Chart: A comprehensive Helm chart that packages the entire NeMo microservices suite and its dependencies, enabling the deployment of the complete NeMo platform. Also referred to as the NeMo platform chart.
NeMo component Helm charts: Individual Helm charts for the NeMo microservices. You can use the component charts to install a subset of NeMo microservices and components tailored to specific requirements. All of the component charts are what’s packaged as subcharts within the NeMo Microservices Helm Chart. These also serve as a convenient reference for advanced configuration when using the NeMo microservices Helm chart.
Overview#
The following sections provide an overview of the NeMo microservices installation process and the deployment scenarios.
Get started by reviewing various deployment scenarios of the NeMo microservices.
Review common information about installing the NeMo microservices using their Helm charts.
Deployment Options#
Deploy the NeMo microservices as a platform or individually.
Deploy the NeMo Microservices as a Platform#
Deploy the entire NeMo microservices as a platform to form a complete data flywheel on your Kubernetes cluster.
Install the parent Helm chart.
Deploy the NeMo microservices as a platform using NVIDIA NIM Operator.
Deploy Individual NeMo Microservices#
Add individual NeMo Microservices to your existing AI workflow.
Install the NeMo Data Store Helm chart.
Deploy the NeMo Entity Store microservice.
Deploy the NeMo Customizer microservice.
Deploy the NVIDIA NeMo Evaluator microservice.
Deploy the NeMo Operator microservice.
Deploy the DGX Cloud Admission Controller for deployment to cloud.
Deploy the NeMo Deployment Management microservice.
Deploy the NIM Proxy microservice.
Deploy the NVIDIA NeMo Guardrails microservice.
Infrastructure Management#
Storage Configuration#
Configure storage for the NeMo microservices.
Configure PostgreSQL databases for the NeMo microservices.
Configure the object storage used for dataset and model files.
Configure the Persistent Volume Claims used by the NeMo Data Store and NeMo Customizer microservices.
Security and Secrets#
Create, access, and update secrets to use across the NeMo microservices.
Manage secrets for pulling container and model images from the NGC registry.
Manage external database secrets used across microservices.
Manage MLflow customizer secrets used across microservices.
Manage Weights & Biases keys used to track and visualize training metrics during fine-tuning.
Other Configurations#
Review and manage other cluster settings.
Review and configure available models, model training methods, and model resources for NeMo Customizer.
Review considerations for single and multi-tenancy configurations.
Review and configure how NeMo Microservices use Open Telemetry for observability.
Review Custom Resource Definitions (CRDs) for training jobs, workloads, and entity handling that NeMo Operator manages.