Create Evaluation Target#

To create a target for an evaluation, send a POST request to the evaluation/targets API. The URL of the evaluator API depends on where you deploy evaluator and how you configure it. For more information, refer to Deploy the NeMo Evaluator Microservice.

Prerequisites#

  • Review the available target types (model, cached_outputs, retriever, rag, rows, and dataset) found in this section.

  • Set your EVALUATOR_BASE_URL environment variable to your evaluator service endpoint:

    export EVALUATOR_BASE_URL="https://your-evaluator-service-endpoint"

To Create an Evaluation Target#

Choose one of the following options to create an evaluation target.

import os
from nemo_microservices import NeMoMicroservices

# Initialize the client
client = NeMoMicroservices(
    base_url=os.environ['EVALUATOR_BASE_URL']
)

# Create an evaluation target
client.evaluation.targets.create(
    type="model",
    name="my-target-model-1",
    namespace="my-organization",
    model={
        "name": "model-MvPLX6aEa1zXJq7YMRCosm",
        "namespace": "default",
        "api_endpoint": {
            "url": "http://nemo-nim-proxy:8000/chat/completions",
            "model_id": "meta/llama-3.1-8b-instruct",
            "format": "nim"
        }
    }
)

print("Target created successfully")

curl -X "POST" "${EVALUATOR_BASE_URL}/evaluation/targets" \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "my-target-model-1",
    "namespace": "my-organization",
    "type": "model",
    "model": {
      "name": "model-MvPLX6aEa1zXJq7YMRCosm",
      "namespace": "default",
      "api_endpoint": {
        "url": "http://nemo-nim-proxy:8000/chat/completions",
        "model_id": "meta/llama-3.1-8b-instruct",
        "format": "nim"
      }
    }
  }'
Example Response
{
    "created_at": "2025-03-19T22:23:28.528061",
    "updated_at": "2025-03-19T22:23:28.528062",
    "id": "eval-target-ABCD1234EFGH5678",
    "name": "my-target-model-1",
    "namespace": "my-organization",
    "type": "model",
    "model": {
        "schema_version": "1.0",
        "id": "model-MvPLX6aEa1zXJq7YMRCosm",
        "type_prefix": "model",
        "namespace": "default",
        "created_at": "2025-03-19T22:23:28.527760",
        "updated_at": "2025-03-19T22:23:28.527762",
        "custom_fields": {},
        "name": "model-MvPLX6aEa1zXJq7YMRCosm",
        "version_id": "main",
        "version_tags": [],
        "api_endpoint": {
            "url": "http://nemo-nim-proxy:8000/chat/completions",
            "model_id": "meta/llama-3.1-8b-instruct",
            "format": "nim"
        }
    },
    "custom_fields": {}
}