`nemo_evaluator.adapters.interceptors.reasoning_interceptor`#

Reasoning interceptor that strips reasoning from responses and tracks reasoning information.

Module Contents#

Adds reasoning information to responses and tracks reasoning metrics.

class nemo_evaluator.adapters.interceptors.reasoning_interceptor.ResponseReasoningInterceptor(params: Params)[source]#

Adds reasoning information to responses and tracks reasoning metrics.

Tracks the following statistics:

Initialization

Initialize the reasoning interceptor.

Args: params: Configuration parameters

class Params(/, **data: Any)[source]#

Bases: nemo_evaluator.logging.BaseLoggingParams

Configuration parameters for reasoning interceptor.

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

intercept_response( resp: nemo_evaluator.adapters.types.AdapterResponse, context: nemo_evaluator.adapters.types.AdapterGlobalContext, ) → nemo_evaluator.adapters.types.AdapterResponse[source]#: Remove reasoning tokens from assistant message content in the response and track reasoning info.

post_eval_hook( context: nemo_evaluator.adapters.types.AdapterGlobalContext, ) → None[source]#: Write collected reasoning statistics to eval_factory_metrics.json.