NeMo Evaluator Quickstart Using Docker Compose#

Run the microservice on your local machine using Docker Compose for experimentation.

Warning

This deployment method has the following limitations:

  • Only custom type evaluation jobs are supported.

  • Models must be deployed externally and accessible to the Evaluator.

Prerequisites#

Before following this deployment guide, ensure that you have:

  • Docker and Docker Compose installed on your system.

  • An NGC API Key for accessing NGC Catalog. Create an NGC API key following the instructions at Generating NGC API Keys. You need the NGC API Key to fetch Docker images required by the NeMo microservices platform.

Set Up#

  1. Export the NGC API Key into your shell environment using the following command:

    export NGC_CLI_API_KEY=<your-ngc-api-key>

  2. Setup the NGC CLI following the instructions at Getting Started with the NGC CLI.

  3. Log in to NVIDIA NGC container registry:

    docker login nvcr.io -u '$oauthtoken' -p $NGC_CLI_API_KEY

  4. Download the Docker Compose configuration from NGC:

    ngc registry resource download-version "nvidia/nemo-microservices/nemo-microservices-quickstart:25.09"
cd nemo-microservices-quickstart_v25.09

  5. Start NeMo Evaluator:

    export NEMO_MICROSERVICES_IMAGE_REGISTRY=nvcr.io/nvidia/nemo-microservices
export NEMO_MICROSERVICES_IMAGE_TAG=25.09
docker compose --profile evaluator up

    The command starts the Evaluator microservice on port 8080.

Verifying the Deployment#

After starting the services, verify everything is working:

  1. Check service status:

    docker compose ps

  2. Verify the service is running:

    curl -fv http://localhost:8080/v1/evaluation/configs

Service Endpoints#

After starting the services with Docker Compose, the following endpoints will be available by default:

  • Evaluator API: http://localhost:8080

    • This is the main endpoint for interacting with the Evaluator microservice.

  • Nemo Data Store HuggingFace Endpoint: http://localhost:3000/v1/hf

    • The Data Store exposes a HuggingFace-compatible API at this endpoint.

    • You can set the HF_ENDPOINT environment variable to this URL if needed for integration or testing.

Stop Evaluator#

To stop Evaluator and its related services, run the following command:

docker compose --profile evaluator down

This command stops and removes the containers started for the Evaluator and its dependencies. You can restart them at any time using the up command.

Include the flag -v or --volumes to remove persistent volumes.

docker compose --profile evaluator down -v

Next Steps#