Filter and Sort Responses from the NVIDIA NeMo Evaluator API#

You can filter and sort results when you query the NVIDIA NeMo Evaluator API. For example, you can list all evaluation jobs, but use a filter to return only jobs that were created today. A sample response that includes information about filtering, sorting, and pagination looks like the following.

{
    "object": "list",
    "data": [
        {
            // item details 
        },
        {
            // item details 
        },
        {
            // item details 
        }
    ],
    "pagination": {
        "page": 1,
        "page_size": 10,
        "current_page_size": 3,
        "total_pages": 1,
        "total_results": 3
    },
    "sort": "created_at",
    "filter": {
        "created_at": {
            "gt": "2025-03-31T00:00:00"
        }
    }
}

Filter Jobs#

You can filter Run and Manage Evaluation Jobs on the following fields. To see a sample response, refer to API.

Field	Type	Valid Values	Example
namespace	string	A namespace specified when a job was created.	`my-organization`
status	enum	One of: `created`, `pending`, `running`, `completed`, `failed`.	`completed`
created_at	datetime	The date that a job was created, in ISO date format. You can use =, gt, lt.	`2025-03-18`
target	string	A target namespace and name (`namespace/name`).	`my-organization/my-model-target-1`
config	string	Config full name (`namespace/name`)	`my-organization/my-gsm8k-config-1`

Example: Filter Jobs by Status#

The following example gets all jobs with the status running.

curl

curl -X "GET" --globoff \
  "${EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[status]=running" \
  -H 'accept: application/json'

Python

endpoint = f"{EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[status]=running"
response = requests.get(endpoint).json()
response

Example: Filter Jobs by Date#

The following example gets all jobs created between March 18, 2025, and March 20, 2025.

curl

curl -X "GET" --globoff \
  "${EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[created_at][gt]=2025-03-18&filter[created_at][lt]=2025-03-20" \
  -H 'accept: application/json'

Python

endpoint = f"{EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[created_at][gt]=2025-03-18&filter[created_at][lt]=2025-03-20"
response = requests.get(endpoint).json()
response

Example: Filter Jobs by Target#

The following example gets all jobs that use a specific target.

curl

curl -X "GET" --globoff \
  "${EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[target]=my-organization/my-model-target-1" \
  -H 'accept: application/json'

Python

endpoint = f"{EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[target]=my-organization/my-model-target-1"
response = requests.get(endpoint).json()
response

Example: Filter Jobs by Config#

The following example gets all jobs that use a specific config.

curl

curl -X "GET" --globoff \
  "${EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[config]=my-organization/my-gsm8k-config-1" \
  -H 'accept: application/json'

Python

endpoint = f"{EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[config]=my-organization/my-gsm8k-config-1"
response = requests.get(endpoint).json()
response

Filter Targets#

You can filter evaluation targets on the following fields. To see a sample response, refer to Create Evaluation Target.

Field	Type	Valid Values	Example
namespace	string	A namespace specified when a target was created.	`my-organization`
name	string	A name specified when a target was created.	`my-model-target-1`
type	string	`cached_outputs`, `model`, `retriever`, `rag`	`model`

Example: Filter Targets by Name#

The following example gets the target with the specified name.

curl

curl -X "GET" --globoff \
  "${EVALUATOR_SERVICE_URL}/v1/evaluation/targets?filter[namespace]=my-organization&filter[name]=my-model-target-1" \
  -H 'accept: application/json'

Python

endpoint = f"{EVALUATOR_SERVICE_URL}/v1/evaluation/targets?filter[namespace]=my-organization&filter[name]=my-model-target-1"
response = requests.get(endpoint).json()
response

Filter Configs#

You can filter evaluation configurations on the following fields.

Field	Type	Valid Values	Example
namespace	string	A namespace specified when a config was created.	`my-organization`
name	string	A name specified when a config was created.	`my-gsm8k-config-1`
type	string	The type of configuration, such as custom, retriever, rag, mt_bench, gsm8k, and others.	`gsm8k`

Example: Filter Configs by Type#

The following example gets all configs of the specified type.

curl

curl -X "GET" --globoff \
  "${EVALUATOR_SERVICE_URL}/v1/evaluation/configs?filter[type]=gsm8k" \
  -H 'accept: application/json'

Python

endpoint = f"{EVALUATOR_SERVICE_URL}/v1/evaluation/configs?filter[type]=gsm8k"
response = requests.get(endpoint).json()
response

Sort API Responses#

You can sort responses in ascending or descending order. To sort results in descending order, put - in front of the sort field.

The following example gets all jobs created after March 18, 2025, sorted by the date that the job was created in descending order.

curl

curl -X "GET" --globoff \
  "${EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[created_at][gt]=2025-03-18&sort=-created_at" \
  -H 'accept: application/json'

Python

endpoint = f"{EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[created_at][gt]=2025-03-18&sort=-created_at"
response = requests.get(endpoint).json()
response

Paginate API Responses#

By default, the API returns page 1, and uses a default page size of 10. You can request a particular page and adjust the page size as shown in the following example.

The following example gets all jobs created after March 18, 2025, sets the page size to 5, and retrieves the information from page 3.

curl

curl -X "GET" --globoff \
  "${EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[created_at][gt]=2025-03-18&page_size=5&page=3" \
  -H 'accept: application/json'

Python

endpoint = f"{EVALUATOR_SERVICE_URL}/v1/evaluation/jobs?filter[created_at][gt]=2025-03-18&page_size=5&page=3"
response = requests.get(endpoint).json()
response