nat.plugins.eval.swe_bench_evaluator.register

Classes

SweBenchEvaluatorConfig

Code patch evaluation for SWE Bench problems.

Functions

register_swe_bench_evaluator(config, builder)

Module Contents

class SweBenchEvaluatorConfig

Bases: nat.data_models.evaluator.EvaluatorBaseConfig

Code patch evaluation for SWE Bench problems.

run_id: str = None

async register_swe_bench_evaluator(

config: SweBenchEvaluatorConfig,

builder: nat.builder.builder.EvalBuilder,

)