`nemo_evaluator.adapters.interceptors.caching_interceptor`#

Caching interceptor with registry support.

Module Contents#

Caching interceptor is special in the sense that it intercepts both requests and responses.

class nemo_evaluator.adapters.interceptors.caching_interceptor.CachingInterceptor(params: Params)[source]#

Caching interceptor is special in the sense that it intercepts both requests and responses.

Initialization

Initialize the caching interceptor.

Args: params: Configuration parameters

class Params(/, **data: Any)[source]#

Bases: nemo_evaluator.logging.BaseLoggingParams

Configuration parameters for caching.

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

intercept_request( req: nemo_evaluator.adapters.types.AdapterRequest, context: nemo_evaluator.adapters.types.AdapterGlobalContext, ) → nemo_evaluator.adapters.types.AdapterRequest | nemo_evaluator.adapters.types.AdapterResponse[source]#: Shall return request if no cache hit, and response if it is. Args: req (AdapterRequest): The adapter request to intercept context (AdapterGlobalContext): Global context containing server-level configuration

intercept_response( resp: nemo_evaluator.adapters.types.AdapterResponse, context: nemo_evaluator.adapters.types.AdapterGlobalContext, ) → nemo_evaluator.adapters.types.AdapterResponse[source]#: Cache the response if caching is enabled and response is successful.