Integrate NeMo Guardrails with NIM from build.nvidia.com#

Set up NeMo Guardrails to use NIM endpoints from build.nvidia.com for cloud-based model inference.

Prerequisites#

An NGC API key with the following key permissions: NGC Catalog, NGC Private Registry, and Public API Endpoints.
- If you need to create a new key, refer to Generating NGC API Keys in the NVIDIA NGC Catalog documentation.
- For more information about the service permissions you can include, refer to Supported NGC Applications and API Key Types in the NVIDIA NGC Catalog documentation.
NIM endpoint URL from your build.nvidia.com account.

Configuration#

To configure NeMo Guardrails to use a NIM endpoint from build.nvidia.com, add the following secret that populates the NIM_ENDPOINT_API_KEY environment variable in the container.

Create a secret using your NGC API key.

$ export NGC_API_KEY="<your-ngc-api-key>"
$ kubectl create secret -n guardrails-ms generic nim-endpoint-api-secret \
    --from-literal=nim-endpoint-api-key=$NGC_API_KEY

Update the values.yaml file with the NIM endpoint configuration:

guardrails:
   guardrails:
      nvcfAPIKeySecretName: nim-endpoint-api-secret
   env:
      NIM_ENDPOINT_URL: <nim-endpoint-url-from-build-nvidia-dot-com>
      NIM_ENDPOINT_API_KEY:
         valueFrom:
            secretKeyRef:
            name: nim-endpoint-api-secret
            key: nim-endpoint-api-key