NeMo Guardrails Deployment Guide#
Deploy the NeMo Guardrails microservice as a standalone component of the NeMo platform with minimal dependencies, without requiring other functional microservices.
Deployment Options#
Choose one of the following deployment options based on your use case.
Deploy the microservice using the Helm chart for production environments.
Quickstart using Docker Compose for local experimentation, development, testing, or lightweight environments.
Deploy Guardrails extension for GKE Inference Gateway.
Deployment with External LLM Endpoints#
Review deployment options for the microservice to connect with external LLM endpoints.
Deploy the microservice using the LLM NIM microservices you have deployed in your cluster separately outside of the NeMo platform.
Deploy the microservice using the LLM NIM endpoints from build.nvidia.com.
Advanced Configurations#
Review advanced deployment and configuration options for the microservice.
Deploy the microservice with high availability by adding configurations for multiple replicas and affinity rules.
Use an external database for NeMo Guardrails configuration store.
Extend the NeMo Guardrails microservice image with additional dependencies and use it in the Helm installation.
Integrate with NeMo Guardrails observability.