Payload Modification

Overview

PayloadParamsModifierInterceptor adds, removes, or modifies request parameters before sending them to the model endpoint.

Tip

Add Request Logging Interceptor to your interceptor chain to verify if your requests are modified correctly.

Configuration

CLI Configuration

--overrides 'target.api_endpoint.adapter_config.params_to_add={"temperature":0.7},target.api_endpoint.adapter_config.params_to_remove=["max_tokens"]'

YAML Configuration

target:
  api_endpoint:
    adapter_config:
      interceptors:
        - name: "payload_modifier"
          enabled: true
          config:
            params_to_add:
              temperature: 0.7
              top_p: 0.9
            params_to_remove:
              - "top_k"              # top-level field in the payload to remove
              - "reasoning_content"  # field in the message to remove
            params_to_rename:
              old_param: "new_param"
        - name: "endpoint"
          enabled: true
          config: {}

Note

In the example above, the reasoning_content field will be removed recursively from all messages in the payload.

Configuration Options

For detailed configuration options, please refer to the nemo_evaluator.adapters.interceptors Python API reference.

Note

The interceptor applies operations in the following order: remove → add → rename. This means you can remove a parameter and then add a different value for the same parameter name.

Use Cases

Parameter Standardization

Ensure consistent parameters across evaluations by adding or removing parameters:

config:
  params_to_add:
    temperature: 0.7
    top_p: 0.9
  params_to_remove:
    - "frequency_penalty"
    - "presence_penalty"

Model-Specific Configuration

Add parameters required by specific model endpoints, such as chat template configuration:

config:
  params_to_add:
    extra_body:
      chat_template_kwargs:
        enable_thinking: false

API Compatibility

Rename parameters for compatibility with different API versions or endpoint specifications:

config:
  params_to_rename:
    max_new_tokens: "max_tokens"
    num_return_sequences: "n"