Create Evaluation Target#

To create a target for an evaluation, send a POST request to the evaluation/targets API. The URL of the evaluator API depends on where you deploy evaluator and how you configure it. For more information, refer to Deploy the NeMo Evaluator Microservice.

Prerequisites#

  • Review the available target types (model, cached_outputs, retriever, rag, rows, and dataset) found in this section.

  • Set your EVALUATOR_SERVICE_URL variable.

    export EVALUATOR_SERVICE_URL="<your evaluator service endpoint>"

    import requests

EVALUATOR_SERVICE_URL = "<your evaluator service endpoint>"

Options#

API#

  1. Perform a POST request to v1/evaluation/targets.

    curl -X "POST" "${EVALUATOR_SERVICE_URL}/v1/evaluation/targets" \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "my-target-model-1",
    "namespace": "my-organization",
    "type": "model",
    "model": {
      "name": "model-MvPLX6aEa1zXJq7YMRCosm",
      "namespace": "default",
      "api_endpoint": {
        "url": "http://nemo-nim-proxy:8000/v1/chat/completions",
        "model_id": "meta/llama-3.1-8b-instruct",
        "format": "nim"
      }
    }
  }'

    import requests

endpoint = f"${EVALUATOR_SERVICE_URL}/v1/evaluation/targets"
payload = {
    "name": "my-target-model-1",
    "namespace": "my-organization",
    "type": "model",
    "model": {
        "name": "model-MvPLX6aEa1zXJq7YMRCosm",
        "namespace": "default",
        "api_endpoint": {
            "url": "http://nemo-nim-proxy:8000/v1/chat/completions",
            "model_id": "meta/llama-3.1-8b-instruct",
            "format": "nim"
        }
    }
}
response = requests.post(endpoint, json=payload, headers={"accept": "application/json"})
print(response.json())

  2. Review the response.

Example Response
{
    "created_at": "2025-03-19T22:23:28.528061",
    "updated_at": "2025-03-19T22:23:28.528062",
    "id": "eval-target-ABCD1234EFGH5678",
    "name": "my-target-model-1",
    "namespace": "my-organization",
    "type": "model",
    "model": {
        "schema_version": "1.0",
        "id": "model-MvPLX6aEa1zXJq7YMRCosm",
        "type_prefix": "model",
        "namespace": "default",
        "created_at": "2025-03-19T22:23:28.527760",
        "updated_at": "2025-03-19T22:23:28.527762",
        "custom_fields": {},
        "name": "model-MvPLX6aEa1zXJq7YMRCosm",
        "version_id": "main",
        "version_tags": [],
        "api_endpoint": {
            "url": "http://nemo-nim-proxy:8000/v1/chat/completions",
            "model_id": "meta/llama-3.1-8b-instruct",
            "format": "nim"
        }
    },
    "custom_fields": {}
}
On this page