Skip to navigation

nemo_gym.web.evaluation_collision

View as MarkdownOpen in Claude

Pure collision planning shared by WebArena preparation and scoring.

Module Contents

Functions

NameDescription
_add_page_image_snapshot-
_add_program_html_snapshot-
_add_snapshot-
_is_fixed_url-
_literal_arg-
_plan_helper_call-
_program_html_collision_key-
_supports_page_image_snapshot-
_supports_program_html_snapshot-
build_collision_planDerive snapshot adapters from a task’s evaluator config.
build_collision_plansBuild collision plans using the selected benchmark tasks as context.
extract_helper_callsReturn helper calls from a func: expression without evaluating it.
has_collision_mitigation-
snapshot_target_key-

Data

HIGH_RISK_HELPERS

REVIEW_HELPERS

API

nemo_gym.web.evaluation_collision._add_page_image_snapshot(
plan: dict[str, typing.Any],
query: dict[str, typing.Any]
) -> None
nemo_gym.web.evaluation_collision._add_program_html_snapshot(
plan: dict[str, typing.Any],
target: dict[str, typing.Any]
) -> None
nemo_gym.web.evaluation_collision._add_snapshot(
plan: dict[str, typing.Any],
adapter: str,
params: typing.Any = {}
) -> None
nemo_gym.web.evaluation_collision._is_fixed_url(
value: typing.Any
) -> bool
nemo_gym.web.evaluation_collision._literal_arg(
node: ast.AST
) -> typing.Any
nemo_gym.web.evaluation_collision._plan_helper_call(
plan: dict[str, typing.Any],
call: dict[str, typing.Any]
) -> None
nemo_gym.web.evaluation_collision._program_html_collision_key(
target: dict[str, typing.Any]
) -> str | None
nemo_gym.web.evaluation_collision._supports_page_image_snapshot(
query: dict[str, typing.Any]
) -> bool
nemo_gym.web.evaluation_collision._supports_program_html_snapshot(
target: dict[str, typing.Any]
) -> bool
nemo_gym.web.evaluation_collision.build_collision_plan(
task_config: dict[str, typing.Any],
program_html_collision_keys: set[str] | None = None
) -> dict[str, typing.Any]

Derive snapshot adapters from a task’s evaluator config.

nemo_gym.web.evaluation_collision.build_collision_plans(
task_configs: list[dict[str, typing.Any]]
) -> list[dict[str, typing.Any]]

Build collision plans using the selected benchmark tasks as context.

Fixed-URL program_html snapshots are only enabled when more than one task in the selected set checks the same mutable target. Most targets use the URL as their collision key; GitLab raw/blob file targets share a key by repo and branch because commits to different files on the same branch can conflict. This avoids snapshot overhead and false negatives for unique mutation tasks while keeping protection for genuinely shared mutable surfaces.

nemo_gym.web.evaluation_collision.extract_helper_calls(
expr: typing.Any
) -> list[dict[str, typing.Any]]

Return helper calls from a func: expression without evaluating it.

nemo_gym.web.evaluation_collision.has_collision_mitigation(
plan: dict[str, typing.Any] | None
) -> bool
nemo_gym.web.evaluation_collision.snapshot_target_key(
target: dict[str, typing.Any]
) -> str
nemo_gym.web.evaluation_collision.HIGH_RISK_HELPERS = {'shopping_get_latest_order_url', 'shopping_get_sku_latest_review_author', 'shop...
nemo_gym.web.evaluation_collision.REVIEW_HELPERS = {'shopping_get_sku_latest_review_author', 'shopping_get_sku_latest_review_rating...