Is this page helpful?

Deploy NeMo Auditor with Docker#

Run the microservice on your local machine using Docker Compose for experimentation.

Prerequisites#

Before following this deployment guide, ensure that you have:

Procedure#

  1. Setup the NGC CLI following the instructions at Getting Started with the NGC CLI.

  2. Set up environment variables:

    $ export NGC_CLI_API_KEY="<your-ngc-api-key>"

  3. Log in to NVIDIA NGC container registry:

    $ docker login nvcr.io -u '$oauthtoken' -p $NGC_CLI_API_KEY

  4. Download the Docker Compose configuration from NGC:

    $ ngc registry resource download-version "nvidia/nemo-microservices/nemo-microservices-quickstart:25.12"
$ cd nemo-microservices-quickstart_v25.12

  5. Start NeMo Auditor:

    $ export NEMO_MICROSERVICES_IMAGE_REGISTRY=nvcr.io/nvidia/nemo-microservices
$ export NEMO_MICROSERVICES_IMAGE_TAG=25.12
$ export NIM_API_KEY="<your-nvidia-api-key>"  # Specify NIM_API_KEY for accessing models from build.nvidia.com and using NIM audit targets.

$ docker compose --profile auditor up --detach --quiet-pull --wait

    You must specify the NIM_API_KEY environment variable. You can specify a dummy value if you do not plan to access models from build.nvidia.com.

    You can also export the OPENAI_API_KEY and REST_API_KEY environment variables if you plan to access models from api.openai.com or use the REST generator in an Auditor target. Refer to Managing Audit Targets for more information about targets.

    NeMo Auditor is available at http://localhost:8080/v1beta1/audit.

Verify the Deployment#

After starting the services, verify everything is working:

  1. Check service status:

    $ docker compose ps

  2. Set AUDITOR_BASE_URL:

    $ export AUDITOR_BASE_URL=http://localhost:8080

  3. Check the auditor servics is running:

    $ curl "${AUDITOR_BASE_URL}/v1beta1/audit/configs"

    Refer to Auditor API for the REST API reference.

Stop the Service#

To stop the microservice:

$ docker compose --profile auditor down

Next Steps#