NeMo Guardrails Microservice Deployment Guide#

Find more information on prerequisites and configuring the key parts of the values.yaml file in the NeMo Guardrails Helm chart.

Prerequisites#

A persistent volume provisioner that uses network storage such as NFS, S3, vSAN, and so on. The guardrails configuration is stored in persistent storage.
Optional: Install NVIDIA GPU Operator if your nodes have NVIDIA GPUs. The demonstration configuration shown on this page does not require NVIDIA GPUs.

Running the Microservice Container#

Set the environment variables with your NGC API key and NVIDIA API key:

$ export NGC_CLI_API_KEY="M2..."
$ export NVIDIA_API_KEY="nvapi-..."

Create a namespace for the microservice:

$ kubectl create namespace guardrails-ms

Add secrets for the NGC API key and NVIDIA API key.

Add a Docker registry secret for downloading the container image from NVIDIA NGC:

$ kubectl create secret -n guardrails-ms docker-registry nvcrimagepullsecret \
    --docker-server=nvcr.io \
    --docker-username='$oauthtoken' \
    --docker-password=$NGC_CLI_API_KEY

Add a generic secret for accessing models from endpoints on NVIDIA API Catalog:

$ kubectl create secret -n guardrails-ms generic nvidia-api-secret \
    --from-literal=nim-endpoint-api-key=$NVIDIA_API_KEY

Search for the NeMo Guardrails Helm chart from the NGC Catalog and select the desired version of the chart. For more information about using Helm charts provided in the NGC Catalog in general, see Helm Charts in the NGC Catalog User Guide.

Download the chart using the helm fetch command with your user name and NGC API key as follows:
```
export NEMO_GUARDRAILS_CHART_VER="<chart-version-to-use>"

helm fetch "https://helm.ngc.nvidia.com/nvidia/nemo-microservices/charts/nemo-guardrails-$NEMO_GUARDRAILS_CHART_VER.tgz" \
   --username='$oauthtoken' \
   --password=$NGC_API_KEY
```

Configure the microservice to use the demonstration configuration.

Save the values from the chart in a file:

$ helm show values nemo-guardrails-$NEMO_GUARDRAILS_CHART_VER.tgz > values.yaml

Edit the values.yaml file with the following changes to enable the demonstration configuration:

env:
  DEMO: True
  NIM_ENDPOINT_API_KEY:
    valueFrom:
      secretKeyRef:
        name: nvidia-api-secret
        key: nim-endpoint-api-key

Start the microservice and port-forward the service:

$ helm install guardrails-ms nemo-microservices/nemo-guardrails \
    --namespace guardrails-ms \
    -f values.yaml

Partial Output

 Get the application URL by running these commands:
   export POD_NAME=...
   export CONTAINER_PORT=...
   echo "Visit http://127.0.0.1:8080 to use your application"
   kubectl port-forward $POD_NAME 8080:$CONTAINER_PORT

Running the export and kubectl commands prints the URL that you can use to interact with the microservice.

Using NFS Configuration Store#

To use an NFS location for the configuration store, you need to add the following to your values.yaml file:

configStore:
  nfs:
    enabled: true
    path: "/path/to/nfs/share"
    server: "nfs-server.example.com"
    mountPath: "/config-store"
    storageClass: "standard"

Important

Make sure the configStore.nfs.path key points to the root of the configuration store directory, which is the directory containing subfolders for each individual Guardrails configuration.