NeMo Guardrails Microservice for Production

The NeMo Guardrails microservice in the NeMo microservices platform is the production-grade, high availability, scalable, end-to-end data flywheel deployment option, available for the NVIDIA AI Enterprise plan.

The NeMo microservices platform provides a fully-featured guardrails microservice that provides the same capabilities as the NeMo Guardrails library through its own API, as the guardrails microservice is built on top of the library.

Capability	NeMo Guardrails Library API Server	NeMo Guardrails Microservice
REST API for guardrailed chat completions	✓	✓
Guardrails configuration management	By managing the config folder structure	Through the platform’s config API endpoints
NeMo microservices platform integration	No need	Required
Production-grade reliability	Limited (self-managed and no high availability support out of the box)	Supported by the platform
Support	Open source community	NVIDIA AI Enterprise plan

When to Use Each Option

Use the following guidelines to decide when to use each option.

NeMo Guardrails Library API Server

The Guardrails API server included in the NeMo Guardrails library is suitable for:

Direct integration of guardrails into your application.
Proof-of-concept deployments.
Development and testing environments.
Evaluating guardrails configurations before production.

For this option, refer to the following topics:

NeMo Guardrails Microservice

The NeMo Guardrails microservice is recommended for:

Production deployments requiring enterprise-grade reliability.
Organizations already using or planning to adopt the NeMo microservices platform.

The NeMo Guardrails microservice operates within the NeMo microservices platform ecosystem and requires other microservices to function. It is not a standalone solution.

For deployment instructions and API documentation for the NeMo Guardrails microservice, refer to NVIDIA NeMo Microservices Documentation.