nemo_evaluator.adapters.types#

Interceptor Interfaces#

class nemo_evaluator.adapters.types.AdapterGlobalContext(output_dir: str, url: str)[source]#

Bases: object

Global context passed to all interceptors containing server-level configuration.

This context contains information that is available throughout the entire request-response cycle and is shared across all interceptors.

property metrics_path: Path#

Path to the eval_factory_metrics.json file.

output_dir: str#
url: str#
class nemo_evaluator.adapters.types.AdapterRequest(
r: flask.wrappers.Request,
rctx: nemo_evaluator.adapters.types.AdapterRequestContext,
)[source]#

Bases: object

r: Request#
rctx: AdapterRequestContext#
class nemo_evaluator.adapters.types.AdapterRequestContext(
cache_hit: bool = False,
cache_key: str | None = None,
request_id: str | None = None,
)[source]#

Bases: object

This is passed with the _whole_ chain from request back to response.

Add here any useful information.

cache_hit: bool = False#
cache_key: str | None = None#
request_id: str | None = None#
class nemo_evaluator.adapters.types.AdapterResponse(
r: requests.Response,
rctx: nemo_evaluator.adapters.types.AdapterRequestContext,
latency_ms: float | None = None,
)[source]#

Bases: object

latency_ms: float | None = None#
r: Response#
rctx: AdapterRequestContext#
exception nemo_evaluator.adapters.types.FatalErrorException[source]#

Bases: Exception

Exception raised when an interceptor encounters a fatal error that should kill the process.

class nemo_evaluator.adapters.types.PostEvalHook[source]#

Bases: ABC

Interface for post-evaluation hooks that run after the evaluation completes.

Post-evaluation hooks are executed after the main evaluation has finished, allowing for cleanup, report generation, or other post-processing tasks.

abstractmethod post_eval_hook(
context: AdapterGlobalContext,
) → None[source]#

Function that will be called by the evaluation system after evaluation completes.

Parameters:

context – Global context containing server-level configuration and evaluation results

Ex.: This is used for report generation, cleanup, metrics collection, etc.

class nemo_evaluator.adapters.types.RequestInterceptor[source]#

Bases: ABC

Interface for providing interception of requests.

abstractmethod intercept_request(
req: AdapterRequest,
context: AdapterGlobalContext,
) → AdapterRequest[source]#

Function that will be called by AdapterServer on the way upstream.

This interceptor can modify the request but must return an AdapterRequest to continue the chain upstream.

Parameters:

  • req – The adapter request to intercept

  • context – Global context containing server-level configuration

Ex.: This is used for request preprocessing, logging, etc.

class nemo_evaluator.adapters.types.RequestToResponseInterceptor[source]#

Bases: ABC

Interface for interceptors that can either continue the request chain or return a response.

abstractmethod intercept_request(
req: AdapterRequest,
context: AdapterGlobalContext,
) → AdapterRequest | AdapterResponse[source]#

Function that will be called by AdapterServer on the way upstream.

If the return type is AdapterRequest, the chain will continue upstream. If the return type is AdapterResponse, the AdapterServer will consider the chain finished and start the reverse chain of responses.

Parameters:

  • req – The adapter request to intercept

  • context – Global context containing server-level configuration

Ex.: the latter case is e.g. how caching interceptor works. For cache miss it will continue the chain, passing request unchanged. For cache hit, it will go for the response.

class nemo_evaluator.adapters.types.ResponseInterceptor[source]#

Bases: ABC

Interface for providing interception of responses.

abstractmethod intercept_response(
resp: AdapterResponse,
context: AdapterGlobalContext,
) → AdapterResponse[source]#

Function that will be called by AdapterServer on the way downstream.

Parameters:

  • resp – The adapter response to intercept

  • context – Global context containing server-level configuration

something