NeMo Guardrails with Docker

Introduction

This guide provides step-by-step instructions for running NeMo Guardrails using Docker. Docker offers a seamless and rapid deployment method for getting started with NeMo Guardrails.

Prerequisites

Ensure Docker is installed on your machine. If not, follow the official Docker installation guide for your respective platform.

Build the Docker Images

1. Clone the repository

Start by cloning the NeMo Guardrails repository:

git clone https://github.com/NVIDIA/NeMo-Guardrails.git nemoguardrails

And change directory into the repository:

cd nemoguardrails

2. Build the Docker image

Build the nemoguardrails Docker image:

docker build -t nemoguardrails .

3. [Optional] Build the AlignScore Server Image

If you want to use AlignScore-based fact-checking, you can also build a Docker image using the provided Dockerfile.

cd nemoguardrails/library/factchecking/align_score
docker build -t alignscore-server .

NOTE: the provided Dockerfile downloads only the base AlignScore image. If you want support for the large model, uncomment the corresponding line in the Dockerfile.

4. [Optional] Build the Jailbreak Detection Heuristics Server Image

If you want to use the jailbreak detection heuristics server, you can also build a Docker image using the provided Dockerfile.

cd nemoguardrails/jailbreak_detection
docker build -t jailbreak_detection_heuristics .

Running using Docker

To run the NeMo Guardrails server using the Docker image, run the following command:

docker run -p 8000:8000 -e OPENAI_API_KEY=$OPENAI_API_KEY nemoguardrails

This will start the NeMo Guardrails server with the example configurations. The Chat UI will be accessible at http://localhost:8000.

NOTE: Since the example configurations use the OpenAI test-davinci-003 models, you need to provide an OPENAI_API_KEY.

To specify your own config folder for the server, you can have to mount your local configuration into the /config path into the container:

docker run \
  -p 8000:8000 \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -v </path/to/local/config/>:/config \
  nemoguardrails

To use the Chat CLI interface, run the Docker container in interactive mode:

docker run -it \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -v </path/to/local/config/>:/config \
  nemoguardrails chat --config=/config --verbose

AlignScore Fact-checking

If one of your configurations uses the AlignScore fact-checking model, you can run the AlignScore server in a separate container:

docker run -p 5000:5000 alignscore-server

This will start the AlignScore server on port 5000. You can then specify the AlignScore server URL in your configuration file:

rails:
  config:
    fact_checking:
      # Select AlignScore as the provider
      provider: align_score
      parameters:
        # Point to a running instance of the AlignScore server
        endpoint: "http://localhost:5000/alignscore_base"