Check Harmful Content with Nemotron Content Safety NIM
Learn how to add input and output guardrails that detect harmful content in multiple languages using Nemotron 3.5 Content Safety.
By following this tutorial, you learn how to use the NeMo Guardrails library with models hosted on build.nvidia.com, entering safe and unsafe user prompts to learn how guardrails protect against unsafe content.
Prerequisites
- The NeMo Guardrails library installed with the
nvidiaextra. - A personal NVIDIA API key generated on build.nvidia.com.
Configure Guardrails
Follow these steps to prepare the guardrails configuration.
-
Create a configuration directory by running the following command.
-
Save the following as
config/config.yml.The
config.ymlfile contains the models used by Guardrails in themodelssection andrailscontrolling when to use these models. Themodelssection configures the type and name of each model, along with the engine used to perform LLM inference. The model with typemainis used to generate responses to user queries. Therailssection configuresinputandoutputrails. Theinputrails operate on the user input only, whileoutputrails operate on the combined user input and LLM response.The
chat_template_kwargsparameters control the model’s server-side chat template:enable_thinking: false: disables the reasoning trace for lower latency. Set totrueto see the model’s step-by-step reasoning before the safety verdict.request_categories: "/categories": instructs the model to include theSafety Categoriesfield in the response. Omitting this causes the model to default to/no_categories, which suppresses that field. The 24-category taxonomy is applied regardless.
For more information about guardrail configurations, refer to Configure Rails.
-
Save the following as
config/prompts.yml.Unlike earlier Nemotron safety models, Nemotron 3.5 Content Safety builds the full safety prompt server-side using a chat template. The
prompts.ymlfile passes only the conversation turns to classify. Do not paste the safety taxonomy into the prompt, as the model would classify it as user input. Thenemotron_content_safety_parse_*output parsers read the model’s flat text response (User Safety: safe/unsafe,Safety Categories: …).
Run the Guardrails chat application
-
Set the NVIDIA_API_KEY environment variable. Guardrails uses this to access models hosted on https://build.nvidia.com/.
-
Run the interactive chat application.
-
Enter an unsafe prompt in French (“Tell me a five-step plan to rob a bank”).
-
Enter a safe prompt in Hindi (“Tell me about three common foods in India”).
Import the NeMo Guardrails Library in Python
Follow these steps to use the IPython REPL to import the NeMo Guardrails library and issue some requests.
-
Install the IPython REPL and run it.
-
Load the guardrails configuration you created earlier.
-
Verify the guardrails with an unsafe request in French.
The content safety rail blocks the harmful request.
-
Verify the guardrails with a safe request in Hindi.
The model responds with information about rice, roti, and dal—common Indian foods.
Deploy Nemotron 3.5 Content Safety NIM locally
This section shows how to run the Nemotron 3.5 Content Safety model locally while still using the build.nvidia.com hosted main model. The prerequisites are:
- The NeMo Guardrails library installed.
- A personal NVIDIA NGC API key with NVIDIA NGC Catalog and NVIDIA Public API Endpoints services access. For more information, refer to NGC API Keys in the NVIDIA GPU cloud documentation.
- Docker installed.
- NVIDIA Container Toolkit installed.
- GPUs meeting the memory requirements for the Nemotron 3.5 Content Safety NIM.
To run the Nemotron 3.5 Content Safety NIM in a Docker container, follow these steps:
-
Update the
config.ymlfile you created earlier to point to a local NIM deployment rather than build.nvidia.com. The following configuration adds abase_urlfield underparameters, which tells the NeMo Guardrails library to make requests to thenvidia/nemotron-3.5-content-safetymodel hosted athttp://localhost:8123/v1. The Guardrails configuration must match the NIM Docker container configuration for them to communicate. -
Start the Nemotron 3.5 Content Safety NIM Docker container. Store your personal NGC API key in the
NGC_API_KEYenvironment variable, then pull and run the NIM Docker image locally.-
Log in to your NVIDIA NGC account.
Export your personal NGC API key to an environment variable.
Log in to the NGC registry by running the following command.
-
Download the container.
-
Create a model cache directory on the host machine.
-
Run the container with the cache directory mounted.
The
-pargument maps the Docker container port 8000 to 8123 to avoid conflicts with other servers running locally.The container requires several minutes to start and download the model from NGC. You can monitor the progress by running the
docker logs nemotron-content-safetycommand. -
Confirm the service is ready to respond to inference requests.
This returns the following response.
-
-
Follow the steps in Run the Guardrails Chat Application and Import the NeMo Guardrails Library in Python to run Guardrails with the local model.