Integrate NeMo Guardrails with LLM NIM Outside of NeMo Platform#

You can set up NeMo Guardrails to use a specific LLM NIM microservice, which you have deployed in your cluster outside of the NeMo platform by using the LLM NIM Helm chart.

Update the values.yaml file to specify the LLM NIM service address to the NIM_ENDPOINT_URL environment variable, and disable NeMo Deployment Management and NIM Proxy.

 1tags:
 2   platform: false
 3   guardrails: true
 4nim:
 5   enabled: true
 6guardrails:
 7   env:
 8      # NIM Proxy service address
 9      # NIM_ENDPOINT_URL: nemo-nim-proxy:8000
10      # A single NIM for LLMs service address
11      NIM_ENDPOINT_URL: <llm-nim-service:port>
12deployment-management:
13   enabled: false
14nim-proxy:
15   enabled: false