`nemo_evaluator.adapters.adapter_config`#

pydantic model nemo_evaluator.adapters.adapter_config.AdapterConfig[source]#

Bases: BaseModel

Adapter configuration with registry-based interceptor support

field discovery: DiscoveryConfig [Optional]#: Configuration for discovering 3rd party modules and directories

field endpoint_type: str = 'chat'#: Type of the endpoint to run the adapter for

field interceptors: list[InterceptorConfig] [Optional]#: List of interceptors to use with their configurations

field log_failed_requests: bool = False#: Whether to log failed requests

field mode: str = 'server'#: Adapter mode: ‘server’ (default) or ‘client’

field post_eval_hooks: list[PostEvalHookConfig] [Optional]#: List of post-evaluation hooks to use with their configurations

classmethod from_legacy_config( legacy_config: dict[str, Any], run_config: dict[str, Any] | None = None, ) → AdapterConfig[source]#

Convert legacy configuration to new interceptor-based format.

Parameters:

legacy_config – Legacy configuration dictionary
run_config – Full run configuration dictionary (optional, used to extract output_dir)

Returns:

AdapterConfig instance with interceptors based on legacy config

Raises:

ValidationError – If legacy_config contains typos or invalid field names

classmethod get_validated_config( run_config: dict[str, Any], ) → AdapterConfig[source]#

Extract and validate adapter configuration from run_config.

Parameters:: run_config – The run configuration dictionary
Returns:: AdapterConfig instance if adapter_config is present in run_config, None otherwise
Raises:: ValueError – If adapter_config is present but invalid

get_interceptor_configs() → dict[str, dict[str, Any]][source]#: Get interceptor configurations as a dictionary

get_post_eval_hook_configs() → dict[str, dict[str, Any]][source]#: Get post-evaluation hook configurations as a dictionary

pydantic model nemo_evaluator.adapters.adapter_config.DiscoveryConfig[source]#

Bases: BaseModel

Configuration for discovering 3rd party modules and directories

field dirs: list[str] [Optional]#: List of directory paths to discover

field modules: list[str] [Optional]#: List of module paths to discover

pydantic model nemo_evaluator.adapters.adapter_config.InterceptorConfig[source]#

Bases: BaseModel

Configuration for a single interceptor

field config: dict[str, Any] [Optional]#: Configuration for the interceptor

field enabled: bool = True#: Whether this interceptor is enabled

field name: str [Required]#: Name of the interceptor to use

pydantic model nemo_evaluator.adapters.adapter_config.LegacyAdapterConfig[source]#

Bases: BaseModel

Legacy adapter configuration parameters (pre-interceptor format).

This model validates legacy configuration dictionaries to catch typos and invalid parameters early, before conversion to the new interceptor format.

Config:

extra: str = forbid

field caching_dir: str | None = None#: Caching directory

field custom_system_prompt: str | None = None#: Custom system prompt

field end_reasoning_token: str = '</think>'#: End reasoning token

field endpoint_type: str = 'chat'#: Endpoint type

field generate_html_report: bool = True#: Generate HTML report

field html_report_size: int | None = 5#: HTML report size

field include_if_reasoning_not_finished: bool | None = None#: Include unfinished reasoning

field include_json: bool = True#: Include JSON in responses

field log_failed_requests: bool = False#: Log failed requests

field logging_aggregated_stats_interval: int = 100#: Logging aggregated stats interval

field max_logged_requests: int | None = None#: Max logged requests

field max_logged_responses: int | None = None#: Max logged responses

field max_saved_requests: int | None = None#: Max saved requests

field max_saved_responses: int | None = None#: Max saved responses

field mode: str = 'server'#: Adapter mode: ‘server’ or ‘client’

field output_dir: str | None = None#: Output directory

field params_to_add: dict[str, Any] | None = None#: Parameters to add

field params_to_remove: list[str] | None = None#: Parameters to remove

field params_to_rename: dict[str, str] | None = None#: Parameters to rename

field process_reasoning_traces: bool = False#: Process reasoning traces

field progress_tracking_interval: int = 1#: Progress tracking interval

field progress_tracking_url: str | None = None#: Progress tracking URL

field save_requests: bool = False#: Save requests to disk

field save_responses: bool = False#: Save responses to disk

field start_reasoning_token: str | None = None#: Start reasoning token

field track_reasoning: bool | None = None#: Track reasoning

field tracking_requests_stats: bool = True#: Track request statistics

field use_caching: bool = True#: Enable caching interceptor

field use_nvcf: bool = False#: Enable NVCF integration

field use_omni_info: bool = False#: Enable omni info processing

field use_progress_tracking: bool = False#: Enable progress tracking

field use_raise_client_errors: bool = False#: Raise client errors

field use_reasoning: bool = False#: Enable reasoning token processing

field use_request_logging: bool = False#: Enable request logging

field use_response_logging: bool = False#: Enable response logging

field use_system_prompt: bool = False#: Enable system prompt modification

class Config[source]#

Bases: object

extra = 'forbid'#

pydantic model nemo_evaluator.adapters.adapter_config.PostEvalHookConfig[source]#

Bases: BaseModel

Configuration for a single post-evaluation hook

Config:

use_enum_values: bool = True

field config: dict[str, Any] [Optional]#: Configuration for the post-evaluation hook

field enabled: bool = True#: Whether this post-evaluation hook is enabled

field name: str [Required]#: Name of the post-evaluation hook to use

class Config[source]#

Bases: object

use_enum_values = True#

nemo_evaluator.adapters.adapter_config#

`nemo_evaluator.adapters.adapter_config`#