Raise Client Error Interceptor

Overview

The Raise RaiseClientErrorInterceptor handles non-retryable client errors by raising exceptions instead of continuing the benchmark evaluation. By default, it will raise exceptions on 4xx HTTP status codes (excluding 408 Request Timeout and 429 Too Many Requests, which are typically retryable).

This interceptor is useful when you want to fail fast on client errors that indicate configuration issues, authentication problems, or other non-recoverable errors rather than continuing the evaluation with failed requests.

Configuration

CLI Configuration

--overrides 'target.api_endpoint.adapter_config.use_raise_client_errors=True'

YAML Configuration

Default Configuration

Raises on 4xx status codes except 408 (Request Timeout) and 429 (Too Many Requests).

target:
  api_endpoint:
    adapter_config:
      interceptors:
        - name: "endpoint"
          enabled: true
          config: {}
        - name: "raise_client_errors"
          enabled: true
          config:
            # Default configuration - raises on 4xx except 408, 429
            exclude_status_codes: [408, 429]
            status_code_range_start: 400
            status_code_range_end: 499

Specific Status Codes

Raises only on specific status codes rather than a range.

target:
  api_endpoint:
    adapter_config:
      interceptors:
        - name: "raise_client_errors"
          enabled: true
          config:
            # Custom configuration - only specific status codes
            status_codes: [400, 401, 403, 404]
        - name: "endpoint"
          enabled: true
          config: {}

Custom Exclusions

Uses a status code range with custom exclusions, including 404 Not Found.

target:
  api_endpoint:
    adapter_config:
      interceptors:
        - name: "raise_client_errors"
          enabled: true
          config:
            # Custom range with different exclusions
            status_code_range_start: 400
            status_code_range_end: 499
            exclude_status_codes: [408, 429, 404]  # Also exclude 404 not found
        - name: "endpoint"
          enabled: true
          config: {}

Configuration Options

For detailed configuration options, please refer to the nemo_evaluator.adapters.interceptors Python API reference.

Behavior

Default Behavior

• Raises exceptions on HTTP status codes 400-499

• Excludes 408 (Request Timeout) and 429 (Too Many Requests) as these are typically retryable

• Logs critical errors before raising the exception

Configuration Logic

1. If status_codes is specified, only those exact status codes will trigger exceptions

2. If status_codes is not specified, the range defined by status_code_range_start and status_code_range_end is used

3. exclude_status_codes are always excluded from raising exceptions

4. Cannot have the same status code in both status_codes and exclude_status_codes

Error Handling

• Raises FatalErrorException when a matching status code is encountered

• Logs critical error messages with status code and URL information

• Stops the evaluation process immediately

Examples

Auth Failures Only

Raises exceptions only on authentication and authorization failures.

config:
  status_codes: [401, 403]

All Client Errors Except Rate Limiting

Raises on all 4xx errors except timeout and rate limit errors.

config:
  status_code_range_start: 400
  status_code_range_end: 499
  exclude_status_codes: [408, 429]

Strict Mode - All Client Errors

Raises exceptions on any 4xx status code without exclusions.

config:
  status_code_range_start: 400
  status_code_range_end: 499
  exclude_status_codes: []

Common Use Cases

• API Configuration Validation: Fail immediately on authentication errors (401, 403)

• Input Validation: Stop evaluation on bad request errors (400)

• Resource Existence: Fail on not found errors (404) for critical resources

• Development/Testing: Use strict mode to catch all client-side issues

• Production: Use default settings to allow retryable errors while catching configuration issues