nat.experimental.test_time_compute.scoring.llm_based_agent_scorer#

Attributes#

logger

Classes#

LLMBasedAgentScorer

Abstract base class for strategy implementations.

Functions#

register_llm_based_agent_scorer(config, builder)

Register the LLM-based agent scorer with the provided configuration and builder.

Module Contents#

logger#
class LLMBasedAgentScorer(
config: nat.data_models.ttc_strategy.TTCStrategyBaseConfig,
)#

Bases: nat.experimental.test_time_compute.models.strategy_base.StrategyBase

Abstract base class for strategy implementations.

This class defines the interface for strategies that can be used in the TTC framework. Concrete strategy classes should implement the methods defined in this class.

llm_bound = None#
async build_components(builder: nat.builder.builder.Builder) → None#

Build the components required for the planner.

supported_pipeline_types() → [nat.experimental.test_time_compute.models.stage_enums.PipelineTypeEnum]#

Return the stage types supported by this selector.

stage_type() → nat.experimental.test_time_compute.models.stage_enums.StageTypeEnum#

Return the stage type of this strategy.

async score_single(
original_prompt: str,
agent_context: str,
item: nat.experimental.test_time_compute.models.ttc_item.TTCItem,
) → float#

Score a single planning item using the LLM.

Args:

original_prompt (str): The original prompt. agent_context (str): The agent context. item (TTCItem): The item to score.

Returns:

float: The score of the item.

async ainvoke(items: list[nat.experimental.test_time_compute.models.ttc_item.TTCItem], original_prompt: str | None = None, agent_context: str | None = None, \*\*kwargs) → list[nat.experimental.test_time_compute.models.ttc_item.TTCItem]#

Score a list of planning items.

Args:

original_prompt (str): The original prompt. agent_context (str): The agent context. items (list[TTCItem]): The list of planning items to score.

Returns:

list[float]: A list of scores corresponding to each planning item.

async register_llm_based_agent_scorer(
config: nat.experimental.test_time_compute.models.scoring_config.LLMBasedAgentScoringConfig,
builder: nat.builder.builder.Builder,
)#

Register the LLM-based agent scorer with the provided configuration and builder.

Args:

config (LLMBasedAgentScoringConfig): The configuration for the LLM-based agent scorer. builder (Builder): The builder instance to use for building components.

Returns:

LLMBasedAgentScorer: The registered LLM-based agent scorer.