NeMo Guardrails Basic Installation Using Helm#

The basic installation sets up the following.

  • NeMo Guardrails as the central service with a PostgreSQL database.

  • NIM Proxy that routes inference requests from NeMo Guardrails to NIM microservices.

  • NeMo Deployment Management that manages NIM microservice deployments.

  • NIM Operator, which is a dependency of the NeMo Deployment Management to deploy NIM microservices.

  • Demo configuration store with sample guardrail configurations. You can use this configuration store for trying out the NeMo Guardrails microservice.

Prerequisites#

The following are the prerequisites for the basic installation.

  • An NGC API key with the following key permissions: NGC Catalog, NGC Private Registry, and Public API Endpoints.

  • (Optional) An external PostgreSQL database. NeMo Guardrails uses a PostgreSQL database to store guardrail configurations. Refer to PostgreSQL for more information.

  • (Optional) Install NVIDIA GPU Operator if your nodes have NVIDIA GPUs.

Installation Steps#

  1. Set the following environment variables with your NGC API key. These environment variables are consumed by the NeMo Microservices Helm Chart to pull your NGC key information while installing. Set both variables to the same NGC API key value.

    $ export NGC_API_KEY="<your-ngc-api-key>"
$ export NVIDIA_API_KEY="<your-ngc-api-key>"

  2. Create a namespace for the microservice:

    $ kubectl create namespace guardrails-ms

  3. Add a Docker registry secret for downloading the NeMo microservice container images from NVIDIA NGC.

    $ kubectl create secret -n guardrails-ms docker-registry nvcrimagepullsecret \
   --docker-server=nvcr.io \
   --docker-username='$oauthtoken' \
   --docker-password=$NGC_API_KEY

  4. Go to the NeMo Microservices Helm Chart page in the NGC Catalog and select the chart version to install. For more information about using Helm charts provided in the NGC Catalog console in general, see Helm Charts in the NGC Catalog User Guide.

    Add the chart repository with the helm repo add command with your user name and NGC API key as follows:

    $ helm repo add nmp "https://helm.ngc.nvidia.com/nvidia/nemo-microservices" \
   --username='$oauthtoken' \
   --password=$NGC_API_KEY

    Then run helm repo update.

  5. Save the default chart values in a file for reference.

    $ helm show values nmp/nemo-microservices-helm-chart > values.yaml

    You can edit the values.yaml file or create a custom values file as needed. For more information about the default values, refer to NeMo Microservices Helm Chart.

  6. Choose one of the following options to install the NeMo Guardrails microservice as a central service and other platform microservices around it.

    • Install with Tags: Choose this option if you want to install the NeMo Guardrails microservice as a central service and other platform microservices around it as configured in the default values file.

    • Update Values File: Choose this option if you have other values you want to update in addition to installing the NeMo Guardrails microservice as a central service.

    Install using tags to install only the necessary services for guardrails as follows.

    $ helm --namespace guardrails-ms \
   install guardrails-ms nmp/nemo-microservices-helm-chart \
   --set tags.platform=false \
   --set tags.guardrails=true

    1. Make the following value overrides in the values.yaml file.

      tags:
   platform: false
   guardrails: true

    2. Run the following command to install the chart with the updated values.

      helm --namespace guardrails-ms \
   install guardrails-ms \
   nmp/nemo-microservices-helm-chart \
   -f values.yaml

Verification#

After installation, verify the deployment:

# Check pod status
$ kubectl get pods -n guardrails-ms

# Check service status
$ kubectl get svc -n guardrails-ms

# Test the service
$ kubectl port-forward -n guardrails-ms svc/nemo-guardrails 7331:7331
$ curl http://localhost:7331/v1/health