> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/labs-voice-agent/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/labs-voice-agent/_mcp/server.

# nemo_voice_agent.evaluation.resume

Resume-state classification utilities shared between the evaluation runner
and the check\_resume.py dry-run script.

## Module Contents

### Functions

| Name                                                                                                   | Description                                                                    |
| ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| [`classify_scenario_resume_state`](#nemo_voice_agent-evaluation-resume-classify_scenario_resume_state) | Classify a scenario directory for resume purposes.                             |
| [`count_agent_llm_messages`](#nemo_voice_agent-evaluation-resume-count_agent_llm_messages)             | Return the number of assistant-role messages in the agent's saved LLM context. |
| [`count_agent_responses`](#nemo_voice_agent-evaluation-resume-count_agent_responses)                   | Return how many LLM responses the agent produced for a scenario.               |

### API

```python
nemo_voice_agent.evaluation.resume.classify_scenario_resume_state(
    scenario_dir: str,
    min_agent_turns: int = 0
) -> typing.Tuple[str, str]
```

Classify a scenario directory for resume purposes.

Returns a `(state, reason)` tuple where `state` is one of:

* `"completed"`  — has a valid `metrics.json` and passes all stall checks;
  the runner will skip it and load metrics from disk.
* `"in_flight"`  — started but not cleanly finished (missing/unreadable
  `metrics.json`, 0 turns, or fewer agent LLM messages
  than `min_agent_turns`); the runner will move it aside
  and re-run it.
* `"fresh"`      — no subdir exists yet; the runner will run it normally.

`reason` is a short human-readable string explaining the classification,
useful for logging and the dry-run check script.

```python
nemo_voice_agent.evaluation.resume.count_agent_llm_messages(
    scenario_dir: str
) -> typing.Optional[int]
```

Return the number of assistant-role messages in the agent's saved LLM context.

Returns None when the file is absent (old runs, crashed before write) or
unreadable — callers treat None as "unknown, skip the check."

NOTE: this file is unreliable — the agent's `bot_logs_agent/llm_context.json`
is frequently saved empty even for scenarios where the agent ran fine (the
context is lost during end-of-scenario retrieval). Prefer
`count_agent_responses` for stall detection; this remains only as a
last-resort fallback for runs whose metrics.json predates `token_usage`.

```python
nemo_voice_agent.evaluation.resume.count_agent_responses(
    scenario_dir: str,
    metrics: typing.Optional[dict] = None
) -> typing.Optional[int]
```

Return how many LLM responses the agent produced for a scenario.

This is the robust stall-detection signal. It prefers the live-recorded
`token_usage.agent.n_calls` from metrics.json — that counter is
accumulated by the bridge *during* the run and survives the agent-context
save bug (see `count_agent_llm_messages`). Resolution order:

1. `metrics["token_usage"]["agent"]["n_calls"]` from the passed-in dict
   (the live runner path hands us the in-memory metrics directly).
2. The same field read from `<scenario_dir>/metrics.json` on disk
   (resume / load-from-disk path).
3. Fallback to `count_agent_llm_messages` for old runs whose metrics.json
   predates the `token_usage` field.

Returns None when no signal is available — callers treat None as
"unknown, skip the check."