nemo_voice_agent.evaluation.scenarios.data.eva_airline

eva_airline scenario package.

Layout: base.py — EvaAirlineBaseScenario + 5 hand-authored seed scenarios. group_Nx.py — auto-scaffolded scenarios for eva sub-flow N.* (generated from eva_airline_dataset.jsonl via scripts/prepare_eva_data/generate_airline_scaffolds.py).

This __init__ re-exports EvaAirlineBaseScenario so the group submodules can keep their canonical import path (from ...eva_airline import EvaAirlineBaseScenario) and triggers the group submodule imports so their @register_eval_scenario decorators fire.

Submodules

Package Contents

Classes

Name	Description
`EvaAirlineBaseScenario`	-

API

class nemo_voice_agent.evaluation.scenarios.data.eva_airline.base.EvaAirlineBaseScenario()

Bases: Scenario

_scenario_db

dict

Load the bound eva scenario JSON. Bridge-side; cached after first read.

agent_actions

Actions

agent_persona

Persona

agent_resources

Resources

agent_task

Task

current_date

str

Scenario’s _current_date from the bound JSON. Single source of truth.

domain

str = 'eva_airline'

eva_id

str = ''

expected_scenario_db

dict

Eva-shipped expected post-run DB state for this scenario.

Sourced from eva_airline_dataset.jsonl’s ground_truth.expected_scenario_db for the matching eva_id. The runner SHA-256-hashes both this and the bridge-pulled final_scenario_db.json to score the scenario on end-state correctness (path-independent — any sequence of agent actions that lands here passes; see evaluation/db_hash.py).

Verified on 2026-05-11: a clean run of scenario 1.1.2 produces a DB whose canonical hash matches this expected state exactly. Hence we use eva’s expected_scenario_db as the ground truth for all airline scenarios rather than hand-authoring NeMo-specific expected states.

Raises KeyError if the eva_id isn’t in the dataset (e.g. a scenario we authored without a corresponding eva entry).

max_duration

= 900

policy

str

Upstream eva 0.1.3 airline instructions (content-verbatim).

success_signals

Base class for airline scenarios ported from eva.

Subclasses set only eva_id (e.g. "1.1.2") — everything else derives:

current_date — read lazily from the bound JSON’s _current_date.
DB seeding — setup_shared_state writes state["db_path"] for the action handler to resolve against EVAL_DATA_ROOT.

Subclasses also declare name, user_persona, user_task, user_actions, reference_answer (a list of expected actions, possibly empty for Q&A-only scenarios), and optionally override agent_actions / agent_resources if the scenario needs domain-specific tweaks.

The toolset is fixed: every airline scenario gets the full eva 15-tool surface plus EndConversationTool. The scenario action list and final DB state are pulled by the bridge at end-of-scenario via the get_scenario_summary RTVI action — no LLM-callable summary tool.

user_resources

Resources

nemo_voice_agent.evaluation.scenarios.data.eva_airline.base.EvaAirlineBaseScenario.get_agent_prompt() -> str

Use eva’s airline policy as the prompt source of truth.

airline_agent.yaml supplies the role and complete policy. NeMo only appends voice-transport and evaluation-harness guidance; policy rules (fees, eligibility, compensation, escalation, and procedure) are never reconstructed locally.

nemo_voice_agent.evaluation.scenarios.data.eva_airline.base.EvaAirlineBaseScenario.setup_shared_state(
    state: dict,
    side: str
) -> None

Seed the agent side with the scenario DB content (inline, not a path).

Symmetric with how the bridge pulls the final DB at end-of-scenario: full content travels both ways. See plan section 6.5 #8.