Supported LLMs#

The NeMo Guardrails library supports a wide range of LLM providers and models. This includes base models, instruct-tuned, and reasoning models. These models can be served locally on the same machine as NeMo Guardrails, or at a remote endpoint accessible from Guardrails over a network. This flexible approach allows Guardrails to be used for a range of applications: from edge deployments on resource-constrained devices, to horizontally-scalable backend clusters.

LLM Types#

Integrating NeMo Guardrails improves safety and security of an Application LLM, which is responsible for generating responses to the end-user. NeMo Guardrails can also use the same Application LLM to run guardrails, simplifying deployments and reducing friction to on-ramp. Two examples of this are self-check rails and dialog rails. Self-check rails use the Application LLM to decide whether a user request or LLM response is safe. Dialog rails use the Application LLM to guide the user through a pre-defined conversational flow.

NeMo Guardrails can also call models for a specific guardrail on behalf of the client. Having guardrail-specific models allows the use of smaller fine-tuned models, which are specialized on the guardrails task. For example the NVIDIA Nemoguard collection of models includes content-safety, topic-control, and jailbreak-detect models. These models can be accessed on build.nvidia.com for rapid prototyping, or on NGC Catalog for deployment with NIM Docker containers.

Application LLM Providers#

The NeMo Guardrails library supports major LLM providers, including:

  • OpenAI

  • Azure OpenAI

  • Anthropic

  • Cohere

  • Google Vertex AI

Self-Hosted#

The NeMo Guardrails library supports the following self-hosted LLM providers:

  • HuggingFace Hub

  • HuggingFace Endpoints

  • vLLM

  • Generic

Providers from LangChain#

The NeMo Guardrails library supports LLM providers from the LangChain Community, including both text completion and chat completion providers. Refer to Chat model integrations in the LangChain documentation. You can also use the nemoguardrails find-providers CLI command to discover available providers.

Embedding Providers#

The NeMo Guardrails library supports the following embedding providers:

  • NVIDIA NIM

  • NVIDIA AI Endpoints

  • FastEmbed

  • OpenAI

  • Azure OpenAI

  • Cohere

  • SentenceTransformers

  • Google

For more information on configuring embedding providers, refer to Embedding Search Providers.