Beginner Tutorial Prerequisites#

Get started with creating a single-node minikube cluster and installing the NeMo microservices platform with the NeMo platform Helm chart.

This page helps cluster administrators, data scientists, and AI application developers deploy the NeMo microservices as an end-to-end platform on a minikube cluster.

After you complete the prerequisite steps, your data scientists and AI application developers can proceed to Beginner Tutorials.

Requirements#

The following requirements are specific to setting up the entire NeMo microservices platform on a minikube cluster for running the Beginner Tutorials. Depending on your environment and what combination of the NeMo microservices you want to deploy, requirements might differ.

System Requirements

A single-node NVIDIA GPU cluster on a Linux host and cluster-admin level permissions.
At least 200 GB of free disk space.
At least two NVIDIA GPUs, A100 80 GB or H100 80 GB, and no other workloads running on them:
- One GPU for machine learning model customization, training, or fine-tuning.
- One GPU for inference.

Software Requirements

NVIDIA Container Toolkit v1.16.2 or higher. Refer to the Installing the NVIDIA Container Toolkit.
NVIDIA GPU Driver 560.35.03 or higher. Refer to Driver Installation Guide.
Kubernetes CLI, kubectl. Refer to Install and Set Up kubectl on Linux in the Kubernetes documentation.
Helm CLI, helm. Refer to the Helm documentation.
Hugging Face CLI. Refer to Command Line Interface in the Hugging Face documentation.

Requirements for the Minikube Cluster

NVIDIA developed and tested this tutorial using minikube and meeting the following prerequisites.

minikube version 1.33 or later.
Docker 27 or later.

This tutorial uses the following minikube features:

minikube ingress.
Standard storage class using host path volumes provided by the default storage provisioner.

The host file system for the host path volumes must support file locking. During customization with NeMo Customizer, NeMo Operator starts an entity handler pod that runs the Hugging Face CLI. The CLI requires a file system, such as EXT4, that supports file locking.

Set Up Minikube Cluster with NeMo Microservices Platform#

You can set up a minikube cluster with the NeMo microservices platform using the installation scripts or manually. Choose one of the following options:

Using Deployment Scripts: Use this option to deploy the NeMo microservices platform using the automated deployment scripts with no manual steps. You can still adjust the deployment parameters by using the demo-values.yaml file.
Installing Manually: Use this option to manually set up the minikube cluster and the NeMo microservices platform with more control over the setup.

Using Deployment Scripts#

Download the following files in a directory on your local machine:
- create-nmp-deployment.sh: NeMo platform deployment script. Use this script to set up a minikube cluster and install the NeMo microservices platform on it.
- destroy-nmp-deployment.sh: Clean-up script. Use this script to delete the minikube cluster and the local files used during the deployment.
- demo-values.yaml: Optional. The create-nmp-deployment.sh script uses this values file or creates one if it doesn’t exist. This values file is for installing the NeMo microservices platform Helm chart, nemo-microservices-helm-chart-25.4.0.tgz, on the minikube cluster for demonstration purposes. Save this file in the same directory as the scripts.

Make the scripts executable.

chmod +x create-nmp-deployment.sh destroy-nmp-deployment.sh

Run the create-nmp-deployment.sh script to set up the minikube cluster and the NeMo microservices platform.

./create-nmp-deployment.sh --helm-chart-url  https://helm.ngc.nvidia.com/nvidia/nemo-microservices/charts/nemo-microservices-helm-chart-25.4.0.tgz

Move onto the Beginner Tutorials to learn how to use the capabilities of the NeMo microservices.
After you’re done with the Beginner Tutorials, run the destroy-nmp-deployment.sh script to delete everything.
```
./destroy-nmp-deployment.sh
```

NeMo Platform Deployment Script Overview#

The create-nmp-deployment.sh script automates the deployment of the NeMo microservices platform on a minikube cluster. Here’s a detailed breakdown of its functionality:

Phase 0: Run pre-flight checks.
- Verifies system requirements.
- Checks required software.
Phase 1: Set up minikube.
- Initializes a fresh minikube cluster with:
  - Docker driver and runtime.
- Unlimited CPU/memory resources.
- GPU access enabled.
- Enables ingress addon.
Phase 2: Create secrets.
- Creates Kubernetes secrets:
  - Docker registry secret (nvcrimagepullsecret).
  - NGC API key secret (ngc-api).
Phase 3: Install the Helm chart.
- Downloads the NeMo Microservices Helm Chart.
- Creates a default demo-values.yaml file if it doesn’t exist.
- Installs the Volcano scheduler.
- Deploys the NeMo platform components.
Phase 4: Verify pod health.
- Monitors pod initialization (30-minute timeout).
- Checks for critical errors (e.g., ImagePullBackOff).
- Collects diagnostic information.
- Validates pod health status.

Phase 5: Configure DNS.

Updates /etc/hosts with DNS entries:

<minikube-ip> nemo.test
<minikube-ip> nim.test
<minikube-ip> data-store.test

Phase 6: Deploy Meta LLaMA NIM.
- Deploys Meta LLaMA 3.1 8B Instruct NIM with the following configuration:
  - Model: llama-3.1-8b-instruct.
  - Storage: 25GB PVC.
  - Resources: 1 GPU.
Phase 7: Wait for NIM readiness.
- Monitors deployment status (15-minute timeout).
- Collects diagnostics if issues occur.
- Ensures NIM reaches READY state.
Phase 8: Verify the NIM endpoint.
- Tests the NIM endpoint responsiveness.
- Validates the models API endpoint.
- Confirms the deployment functionality.

Script Usage#

./create-nmp-deployment.sh [OPTIONS]

Options:
  --helm-chart-url URL    Override default helm chart URL
  --values-file FILE      Specify additional values file(s)
  --help                  Show help message

Installing Manually#

If you want to manually set up the minikube cluster and the NeMo microservices platform, follow the steps in the following sections.

Start Minikube#

Download minikube following the minikube start guide in the minikube documentation.
Refer to the instructions for Using NVIDIA GPUs with minikube and follow the steps until you reach the command to start minikube. When you get to that point, use the following command to ensure that there is enough RAM and CPU.
```
minikube start \
   --driver docker \
   --container-runtime docker \
   --cpus no-limit \
   --memory no-limit \
   --gpus all
```
Enable the minikube ingress addon.
```
minikube addons enable ingress
```

Install the NeMo Microservices Platform#

Use the NVIDIA NeMo Microservices Helm Chart to install the NeMo microservices platform.

Create an NGC API key following the instructions at Generating NGC API Keys.
Export the key into your shell environment using the following command:
```
export NGC_API_KEY=<your-ngc-api-key>
```

Set up NGC secrets in the default namespace of the cluster using the following command:

kubectl create secret \
   docker-registry nvcrimagepullsecret \
   --docker-server=nvcr.io \
   --docker-username='$oauthtoken' \
   --docker-password=$NGC_API_KEY

kubectl create secret generic ngc-api \
   --from-literal=NGC_API_KEY=$NGC_API_KEY

Download the NeMo Microservices Helm Chart:

helm fetch --untar https://helm.ngc.nvidia.com/nvidia/nemo-microservices/charts/nemo-microservices-helm-chart-25.4.0.tgz \
   --username='$oauthtoken' \
   --password=$NGC_API_KEY

Download the demo-values.yaml file.

Note

The demo-values.yaml file provides a basic cluster setup with default values. For production environments, you need additional configurations for:

Database disaster recovery
Multi-node training support in Volcano
Persistent volume claims (PVC) with proper storage classes The demo-values.yaml file is not suitable for production environments. For production-grade deployments, see the Admin Setup section.

Install Volcano scheduler before installing the chart:

VOLCANO_VERSION=$(yq '.dependencies[] | select(.name=="volcano") | .version' nemo-microservices-helm-chart/Chart.yaml)
kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/v${VOLCANO_VERSION}/installer/volcano-development.yaml

Install the chart:
```
helm --namespace default install \
   nemo nemo-microservices-helm-chart-25.4.0.tgz \
   -f demo-values.yaml
```
The pods require approximately 30 minutes to download images, start the containers, and establish stable communication. During this time, it is normal for pods to be in a pending or restarting state.
Verify that the pods are in the ready state:
```
kubectl get pods
```
Confirm that pods with a running status display 1/1 or 2/2. If pods do not enter the Running or Completed state after 30 minutes, run kubectl events to check for errors.

If the pods fail to stabilize, you can investigate image or Kubernetes issues by running kubectl events. If you want to diagnose issues on a specific deployment, add --for deployment/<deployment name> to target the deployment. You can also investigate software issues using kubectl logs <name of concerning pod>.

Configure DNS Resolution#

Display the ingress resources:

kubectl get ingress

The following is an example output.

NAME                            CLASS    HOSTS                      ADDRESS   PORTS   AGE
nemo-microservices-helm-chart   <none>   nim.test,data-store.test             80      34m

Export an environment variable with the accessible IP address of your ingress controller:
```
export NEMO_HOST=$(minikube ip)
```
Add host name entries in the /etc/hosts file for the *.test ingress hosts to use the accessible IP address. Make a backup of the /etc/hosts file before you make the changes.
```
sudo cp /etc/hosts /etc/hosts.bak
echo -e "$NEMO_HOST nemo.test\n$NEMO_HOST nim.test\n$NEMO_HOST data-store.test\n" | sudo tee -a /etc/hosts
```
To learn more about how the hosts and their default path rules are configured, refer to Ingress Setup for Production Environment.

Tip

If you complete the steps in this section, the minikube cluster is ready with the NeMo microservices platform installed. Proceed to the Beginner Tutorials to learn how to use the capabilities of the NeMo microservices.

Clean Up#

After you’re done with the Beginner Tutorials, delete the minikube cluster to clean up.

Warning

This deletes the minikube cluster and all the NeMo platform setup and the resources associated within it. Do not run this command unless you are done with the tutorial and want to delete the minikube cluster.

minikube delete

Recover the /etc/hosts file from the backup to remove the host name entries for the *.test ingress hosts.

sudo cp /etc/hosts.bak /etc/hosts

Deploy the NeMo Microservices Platform to a Production-Grade Kubernetes Cluster#

For more information about deploying the NeMo microservices platform to a production-grade Kubernetes cluster, proceed to the Admin Setup section.