Guardrail Tutorials#
Use the following tutorials to learn how to accomplish common guardrail tasks using the NeMo Guardrails API.
Tip
The tutorials reference NMP_BASE_URL, whose value depends on the ingress in your cluster. Before starting a tutorial, complete the ModelProvider setup below.
Prerequisite: Configure a ModelProvider#
The tutorials use NIMs hosted on build.nvidia.com.
Note
The platform pre-configures a system/nvidia-build model provider during startup.
This provider routes inference requests to models hosted on build.nvidia.com using the API base URL https://integrate.api.nvidia.com
and the NGC API key with Public API Endpoints permissions provided during deployment (automatically saved as the built-in system/ngc-api-key secret).
You can verify this provider exists by running nmp inference providers list --workspace system.
The tutorials in these docs use this provider for inference, but you can alternatively create your own and use it instead.
Once the ModelProvider is configured, use Model Entity references (workspace/model_name format) as the model in your guardrail configurations. Internally, the Inference Gateway service routes requests to the Model Provider. Refer to Model Routing for more details.
guardrails_config = {
"models": [
{
"type": "content_safety",
"engine": "nim",
"model": "system/nvidia-llama-3-1-nemotron-safety-guard-8b-v3",
}
],
# ... rest of config
}
Deploy NemoGuard NIMs in your environment
Use Content Safety checks to detect and block harmful content
Configure parallel rails for input and output guardrails.
Safety checks for multimodal data with NeMo Guardrails API.
Configure checks for SQL, XSS, template, and code injection.