Verification Patterns

Verification is how an environment evaluates agent behavior and computes a score. Every environment implements some form of verification — the pattern you choose depends on your task.

Equivalence Match

Compare the agent’s output to a known reference answer.

Execution and State Match

Execute the agent’s actions, such as tool calls or generated code, and verify the output or resulting state.

LLM-as-Judge

Prompt an LLM to evaluate the agent’s output against rubrics, instructions, or reference answers.

Reward Model

Use an LLM trained on human preferences to score outputs for alignment, such as RLHF reward modeling.

coming soon