> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/skills/skillevaluator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/skills/skillevaluator/_mcp/server.

# Performance Dashboard

> Compare saved Tier 3 runs by skill, model, harness, score, tokens, time, and recorded cost in the optional local dashboard.

Use the dashboard to compare existing evaluation results without running new
agent trials. It reads saved JSON reports and retained Tier 3 runs; missing
measurements remain unavailable.

## Start the dashboard

Install the optional Python dashboard and open saved JSON reports or individual
retained Tier 3 run directories:

```bash
uv tool install --python 3.13 --reinstall "skillevaluator[all,dashboard] @ git+https://github.com/NVIDIA/SkillEvaluator.git"
skillevaluator dashboard reports/run-a.json reports/run-b.json
skillevaluator dashboard ./my-skill/evals/results/<run-id>
# Start with an empty dashboard and upload several JSON reports in the browser:
skillevaluator dashboard
```

From a checkout, use `uv run --extra dashboard skillevaluator dashboard ...`.
For live reload while developing the dashboard, run
`PYTHONPATH=src uv run --extra dashboard streamlit run src/skillevaluator/reporting/dashboard_ui/skillevaluator_dashboard.py -- PATH`;
`PYTHONPATH=src` lets Streamlit watch the package modules for changes.
The server runs at `http://127.0.0.1:8501`, or the next free port when 8501 is
busy; pass `--port 8502` to require a specific port. Use `--no-browser` to
suppress opening a browser, and Ctrl+C to stop it.
The dashboard runs from its own package directory, so Streamlit settings or
secrets in your project's `.streamlit/` folder are not applied. It accepts
browser connections only for 127.0.0.1 and localhost.
Browser uploads accept up to 16 MB (16,000,000 bytes) per file. Local JSON files
accept up to 16 MiB (16,777,216 bytes); larger files produce a notice while other
valid reports remain available.

## Compare saved runs

Filter the sortable tables by skill, model, harness, recorded harness features,
condition, and execution status. The status and condition filters narrow the
performance table only, so comparisons keep both arms; the other filters also
remove arms from comparisons. Download the visible performance or comparison
rows as CSV. Each performance row is one run, agent (harness and model),
recorded feature configuration, and condition.
Per-trial columns are means over the row's measured trials and stay blank when
any trial is unmeasured. They help compare rows with different trial counts,
but runs on different datasets are still not like-for-like.
The comparison table reports **with skill minus without skill** differences,
with percentage changes for usage relative to without skill. A negative usage
difference means less usage; a positive score difference means higher performance. Comparisons stay within
the same report and agent and require successful conditions with matching trial
coverage, the same dataset digest, and the same attempt policy.
The score difference preserves the evaluator's recorded Skill Lift, including
negative, zero, or unavailable values. Cases judged by different models are
excluded from lift, so it can differ from subtracting the displayed arm scores.
Pass-rate and resource differences can remain available when score lift is
unavailable. If every positive with-skill trial has confirmed zero activation,
the comparison is **inconclusive** and all differences stay unavailable. See
[Skill Lift](/skills/skillevaluator/reports#skill-lift) for these eligibility rules.

When a pair's differences stay blank, the **Comparison reason** column says why.
Repeated attempts pair by case, as in the HTML report: each arm must record the
same number of attempts for every case, and each attempt must have its own
trial ID. A trajectory above the report loader's size bound leaves that trial's
token counts unavailable in canonical JSON but does not block score comparisons.
Truncation limited to diagnostic details, such as evidence, raw reward fields,
or non-best agent details, also leaves comparisons available, with a notice.
Truncation that drops trials or other trial artifacts, or that is not
explained, keeps comparisons unavailable.

## Reload reports

The dashboard reads each input once and reuses it while you filter or switch
views. Rewriting a JSON file passed on the command line, or adding an entry
directly inside a directory input, is picked up on the next interaction. Edits
deeper inside a directory input (for example a run that is still writing trials)
or in reports a catalog references appear after **Reload reports** in the
sidebar; a browser refresh does not re-read them. Overlapping inputs, such as a
results directory and a run inside it, are loaded once. Byte-identical uploads
are skipped with a notice.

## Measurement limits

Missing measurements stay unavailable, and cost is shown only when recorded in
USD. Canonical Tier 3 JSON records each single-step trial's agent execution time
(`agent_seconds`) when Harbor's timestamps carry a timezone. Timestamps without a
timezone are treated as unavailable, as in the HTML report's
[measured agent resources](/skills/skillevaluator/reports#measured-agent-resources). Agent time excludes
environment setup and verification. Canonical JSON never records USD cost; open
the retained run directory to show Harbor's recorded `agent_result.cost_usd` and
multi-step totals. For retained runs whose trajectory lacks token totals or
cannot be read, the dashboard falls back to Harbor's `agent_result` token
counters and keeps the trial's agent time and cost.
Exports written before v0.5.0 could encode missing token counters as zero, so
when a canonical JSON report's `evaluator_version` is earlier than 0.5.0 the
dashboard shows its zero token counters as unavailable and adds a notice; open
the retained run directory to recover recorded counters.
Cached tokens are shown separately and are not added again
to prompt plus completion tokens. No pricing estimates or new agent evaluations
are performed. Adverse conditions and harness features are displayed when
explicitly recorded; the dashboard does not run these experiments.

Since v0.5.0, canonical JSON collapses multi-step trials without per-trial token
or time totals, so they show as unmeasured. Older exports that repeat a trial ID
across reward rows show an ambiguity notice. Open the retained run directory to
recover totals from authoritative step results.

## Recorded harness features

Features are experiment metadata, not inferred from the harness name. Current
SkillEvaluator commands do not record this field; add it to the canonical JSON
or to `run_config.json` by hand. In a canonical Tier 3 JSON payload, an agent or
condition can include:

```json
"dashboard_metadata": {
  "harness_features": ["claude-rules:enabled", "memory:disabled"]
}
```

`harness_features` can be a list of strings, a non-empty string, or a JSON
object. For canonical JSON the condition's value wins over the agent's, which
wins over the report's. For retained runs the order is the condition's
`summary.json`, then the agent entry in `run_config.json`, then the root of
`run_config.json`. Edit the run directory's `run_config.json`; the copy embedded
in `result.json` is used only when that file cannot be read. An empty feature
list records that no features were enabled and is shown as **none (recorded)**;
an absent field is displayed as **unknown**. Differing recorded models or
feature configurations are kept separate when pairing conditions.

An explicitly imported adverse condition uses `conditions.adverse`,
`trials_adverse`, `num_trials_adverse`, and `adverse` for its score in the same
agent payload. It appears in the performance table; usage differences are
calculated for with/without pairs only. Adverse conditions are import-only
metadata: no evaluator command produces them, and the dashboard does not run them.