`nemo_evaluator.adapters.adapter_config`#

Submodule responsible for the configuration related specifically to adapters.

For the visibility reasons, we don’t expose adapter configuration via CLI. All the adaptor config comes from the framework configuration yaml under

target:
  api_endpoint:
    adapter_config:
      discovery:
        modules: ["mod.a.b.c", ...]
        dirs: ["/some/path"]
      interceptors: []
      post_eval_hooks: []
      endpoint_type: "chat"  # default: "chat"
      caching_dir: "/some/dir"  # default: null
      generate_html_report: true  # default: true
      log_failed_requests: false  # default: false
      tracking_requests_stats: true  # default: true
      html_report_size: 5  # default: 5

This module merely takes such a dict and translates it into a typed dataclass.

Module Contents#

Classes#

`AdapterConfig`	Adapter configuration with registry-based interceptor support
`DiscoveryConfig`	Configuration for discovering 3rd party modules and directories
`InterceptorConfig`	Configuration for a single interceptor
`PostEvalHookConfig`	Configuration for a single post-evaluation hook

API#

class nemo_evaluator.adapters.adapter_config.AdapterConfig(/, **data: Any)[source]#

Bases: pydantic.BaseModel

Adapter configuration with registry-based interceptor support

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

caching_dir: str | None#: ‘Field(…)’

discovery: nemo_evaluator.adapters.adapter_config.DiscoveryConfig#: ‘Field(…)’

endpoint_type: str#: ‘Field(…)’

classmethod from_legacy_config( legacy_config: dict[str, Any], run_config: dict[str, Any] | None = None, ) → nemo_evaluator.adapters.adapter_config.AdapterConfig[source]#

Convert legacy configuration to new interceptor-based format.

Args: legacy_config: Legacy configuration dictionary run_config: Full run configuration dictionary (optional, used to extract output_dir)

Returns: AdapterConfig instance with interceptors based on legacy config

generate_html_report: bool#: ‘Field(…)’

get_interceptor_configs() → dict[str, dict[str, Any]][source]#: Get interceptor configurations as a dictionary

classmethod get_legacy_defaults() → dict[str, Any][source]#: Get default values for legacy configuration parameters.

get_post_eval_hook_configs() → dict[str, dict[str, Any]][source]#: Get post-evaluation hook configurations as a dictionary

classmethod get_validated_config( run_config: dict[str, Any], ) → nemo_evaluator.adapters.adapter_config.AdapterConfig[source]#

Extract and validate adapter configuration from run_config.

Args: run_config: The run configuration dictionary

Returns: AdapterConfig instance if adapter_config is present in run_config, None otherwise

Raises: ValueError: If adapter_config is present but invalid

html_report_size: int | None#: ‘Field(…)’

interceptors: list[nemo_evaluator.adapters.adapter_config.InterceptorConfig]#: ‘Field(…)’

log_failed_requests: bool#: ‘Field(…)’

post_eval_hooks: list[nemo_evaluator.adapters.adapter_config.PostEvalHookConfig]#: ‘Field(…)’

tracking_requests_stats: bool#: ‘Field(…)’

class nemo_evaluator.adapters.adapter_config.DiscoveryConfig(/, **data: Any)[source]#

Bases: pydantic.BaseModel

Configuration for discovering 3rd party modules and directories

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

dirs: list[str]#: ‘Field(…)’

modules: list[str]#: ‘Field(…)’

class nemo_evaluator.adapters.adapter_config.InterceptorConfig(/, **data: Any)[source]#

Bases: pydantic.BaseModel

Configuration for a single interceptor

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

config: dict[str, Any]#: ‘Field(…)’

enabled: bool#: ‘Field(…)’

name: str#: ‘Field(…)’

class nemo_evaluator.adapters.adapter_config.PostEvalHookConfig(/, **data: Any)[source]#

Bases: pydantic.BaseModel

Configuration for a single post-evaluation hook

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

class Config[source]#

use_enum_values#: True

config: dict[str, Any]#: ‘Field(…)’

enabled: bool#: ‘Field(…)’

name: str#: ‘Field(…)’

nemo_evaluator.adapters.adapter_config#

Module Contents#

Classes#

API#

`nemo_evaluator.adapters.adapter_config`#