V2 API Migration Guide#

This guide helps you migrate from the v1 to v2 Evaluator API.

Note

Migration Timeline: The v1 API will remain fully supported during the v2 transition period. You can migrate at your own pace.

Warning

v2 API Preview: The v2 API is available for testing and feedback but is not yet recommended for production use. Breaking changes may occur before the stable release.

Breaking Changes Summary#

Removed Endpoints#

The following v1 endpoints have been removed in v2. Their functionality has been consolidated into other endpoints:

v1 Endpoint	v2 Replacement	Notes
`GET /v1/evaluation/jobs/{id}/status-details`	`GET /v2/evaluation/jobs/{id}` `GET /v2/evaluation/jobs/{id}/status`	Status details included in job details response or `/status` endpoint.

Changed Endpoints#

v1 Endpoint	v2 Endpoint	Breaking Changes
`GET /v1/evaluation/jobs/{id}/logs`	`GET /v2/evaluation/jobs/{id}/logs`	Major: Returns JSON with pagination instead of ZIP file
`GET /v1/evaluation/jobs/{id}/download-results`	`GET /v2/evaluation/jobs/{id}/results/artifacts/download`	New path structure and downloads a gzip instead of a zip artifact.
`GET /v1/evaluation/jobs/{id}/results`	`GET /v2/evaluation/jobs/{id}/results/evaluation-results/download`	New path structure

Request Format Changes#

Job Creation#

The most significant change is in job creation, which now requires additional fields and uses a spec envelope.

v1 Format

{
  "namespace": "my-organization",
  "target": "my-organization/my-target",
  "config": "my-organization/my-config"
}

v2 Format

Example job with in-line target and config (secrets supported).

{
  // New required envelope
  "spec": {
    "target": {
      // example target definition
      "type": "model",
      "model": {}
    },
    "config": {
      // example config definition
      "type": "custom",
      "params": {}
    }
  }
}

Example job referencing config and target created with v1 API. Secrets are not supported, see the following V2 Secrets section.

{
  // New required envelope
  "spec": {
    "target": "my-organization/my-target",
    "config": "my-organization/my-config"
  }
}

Key Changes:#

Spec envelope: Target and config must be wrapped in a required spec object
Endpoint URL: Use /v2/evaluation/jobs instead of /v1/evaluation/jobs

V2 Secrets#

Jobs created with the v2 API cannot use configuration or targets from the v1 API containing API keys. This will result in missing secrets and the job may fail if the secrets are needed.

To securely use API keys for jobs with the v2 API, the secrets must be defined in-line with the job definition.

{
  "spec": {
    "target": {
      "type": "model",
      "model": {
        // API key is supported
      }
    },
    "config": {
      "type": "custom",
      "params": {
        // API keys are supported
      }
    }
  }
}

Response Format Changes#

Job Status Consolidation#

v1 Approach

Multiple endpoints for status information

GET /v1/evaluation/jobs/{id} - Basic job info
GET /v1/evaluation/jobs/{id}/status - Status only
GET /v1/evaluation/jobs/{id}/status-details - Detailed status

# v1 - Multiple API calls needed
job = client.evaluation.jobs.retrieve(job_id)
status = client.evaluation.jobs.get_status(job_id)
status_details = client.evaluation.jobs.get_status_details(job_id)

v2 Approach

Single consolidated endpoint

GET /v2/evaluation/jobs/{id} - All information including status and status_details

# v2 - Single API call
job = client.evaluation.jobs.retrieve(job_id)
# job.status and job.status_details are included

Log Format Changes#

This is the most significant breaking change in v2.

v1 Approach

ZIP file download

curl "${EVALUATOR_BASE_URL}/v1/evaluation/jobs/{id}/logs" -o logs.zip
unzip logs.zip
cat logs/job.log

v2 Approach

JSON response with pagination

curl "${EVALUATOR_BASE_URL}/v2/evaluation/jobs/{id}/logs" | jq -r '.data[].message'

v2 Response Format:

{
  "data": [
    {
      "id": 4362,
      "job_id": "job-dq1pjj6vj5p64xaeqgvuk4",
      "job_step": "evaluation",
      "job_task": "64976129addc499f88071823fcc48f30",
      "message": "Log message here\n",
      "timestamp": "2025-09-08T19:20:44.799146"
    }
  ],
  "next_page": "cursor",
  "prev_page": null,
  "total": 373
}

Result Access Changes#

v1 Paths:

Download results: /v1/evaluation/jobs/{id}/download-results
Get results: /v1/evaluation/jobs/{id}/results

v2 Paths:

Download artifacts: /v2/evaluation/jobs/{id}/results/artifacts/download
Get evaluation results: /v2/evaluation/jobs/{id}/results/evaluation-results/download
List available results: /v2/evaluation/jobs/{id}/results

Migration Examples#

Job Creation Migration#

Before (v1)

job = client.evaluation.jobs.create(
    namespace="my-organization",
    target="my-organization/my-target",
    config="my-organization/my-config"
)

After (v2)

job = client.v2.evaluation.jobs.create(
    spec={
        "target": "my-organization/my-target",
        "config": "my-organization/my-config"
    }
)

Status Monitoring Migration#

Before (v1)

# Multiple API calls
job = client.evaluation.jobs.retrieve(job_id)
status = client.evaluation.jobs.get_status(job_id)
status_details = client.evaluation.jobs.get_status_details(job_id)

print(f"Status: {status}")
print(f"Details: {status_details}")

After (v2)

# Single API call
job = client.v2.evaluation.jobs.retrieve(job_id)

print(f"Status: {job.status}")
print(f"Details: {job.status_details}")
print(f"Errors: {job.error_details}")

Log Processing Migration#

Before (v1)

# Download ZIP file
logs_zip = client.evaluation.jobs.download_logs(job_id)
with open(f"{job_id}_logs.zip", 'wb') as f:
    f.write(logs_zip)

# Extract and process
import zipfile
with zipfile.ZipFile(f"{job_id}_logs.zip", 'r') as zip_ref:
    zip_ref.extractall("logs")
    
with open("logs/job.log", 'r') as f:
    for line in f:
        print(line.strip())

After (v2)

# Get JSON logs with pagination
logs_response = client.v2.evaluation.jobs.logs.list(job_id)

# Process logs directly
for log_entry in logs_response.data:
    print(f"[{log_entry.timestamp}] {log_entry.message.strip()}")

# Handle pagination
while logs_response.next_page:
    logs_response = client.evaluation.jobs.logs.list(
        job_id, 
        page_cursor=logs_response.next_page
    )
    for log_entry in logs_response.data:
        print(f"[{log_entry.timestamp}] {log_entry.message.strip()}")

Results Download Migration#

Before (v1)

# Download results
results_zip = client.evaluation.jobs.download_results(job_id)
with open(f"{job_id}_results.zip", 'wb') as f:
    f.write(results_zip)

After (v2)

# List available results
available_results = client.v2.evaluation.jobs.results.list(job_id)
print(f"Available results: {[r.result_name for r in available_results.data]}")

# Download specific result types
artifacts_zip = client.v2.evaluation.jobs.results.artifacts.retrieve(job_id)
artifacts_zip.write_to_file("evaluation_artifacts.tar.gz")

eval_results = client.v2.evaluation.jobs.results.evaluation_results.retrieve(job_id)
with open("evaluation_results.json", "w") as f:
    f.write(eval_results.model_dump_json(indent=2, exclude_none=True))