Quickstart

Start with a complete example included in the repository. Choose a keyless quality check, or add NVIDIA Build to get feedback from a model.

Get the example

Install uv and Git, then clone the repository:

git clone --depth 1 https://github.com/NVIDIA/SkillEvaluator.git
cd SkillEvaluator

Run the commands below from this checkout. uv provisions Python 3.13 and installs the dependencies needed for the selected path. The bundled text-analyzer skill includes its SKILL.md and a working text-analysis script, so you can evaluate a complete skill before creating your own.

Run without an LLM

This path runs offline after installation. It needs no API key, model, security scanner, GPU, or Docker runtime.

uv run --python 3.13 skillevaluator quality-check \
  src/skillevaluator/tier3/reference_skills/text-analyzer \
  -r cli -r json -o reports/first-quality

The terminal shows a score from 0–100, an A–F grade, and individual findings. With the current quality checks, the bundled example scores 84.5/100 (B) with 7 findings. It passes the default minimum score of 70 and exits 0. The same result is saved to:

reports/first-quality/skillevaluator-quality.json

This checks static quality across correctness, discoverability, reliability, and efficiency. It does not run the security scanners or measure an agent’s performance. The report’s publication status remains INCOMPLETE because a quality score alone does not supply all required evaluation evidence. See Reports & Results for the full output contract.

Use NVIDIA Build

Add model-backed rubric feedback for the same example. NVIDIA Build hosts the model, so this first model call needs no local GPU or Docker runtime.

Eligible NVIDIA Developer Program accounts have free access to NVIDIA-hosted API endpoints for prototyping. Check the NVIDIA access FAQ and your account for current eligibility, model availability, and rate limits.

1. Open NVIDIA Build, sign in or create an NVIDIA account, and complete the Developer Program enrollment prompts.
2. Open a model page, choose Get API Key, then generate and copy your key. The API Catalog quickstart walks through this flow.
3. Set the provider and key in the same terminal. Replace the example key with the key you copied:

export SKILL_EVAL_LLM_PROVIDER=nv_build
export NVIDIA_API_KEY='nvapi-...'

Run the rubric evaluation with the llm extra:

uv run --python 3.13 --extra llm skillevaluator rubric-eval \
  src/skillevaluator/tier3/reference_skills/text-analyzer \
  -r cli -r json -o reports/first-build

The terminal shows the model’s rubric scores and feedback, and saves the report to:

reports/first-build/skillevaluator-rubric.json

Scores can vary between model calls. This evaluates the skill’s instructions; it does not run an agent with the skill. For model overrides or another provider, see Providers & Credentials.

Where to go next

Check your own skill

Replace the sample path with a directory containing your SKILL.md. Keep the report output outside that directory:

uv run --python 3.13 skillevaluator quality-check ./my-skill \
  -r cli -r json -o reports/my-skill-quality

Add full static validation

Full static Tier 1 coverage needs the security extra and the separate Semgrep, SkillSpector, and Gitleaks executables. Follow scanner setup, then run:

uv run --python 3.13 --extra security skillevaluator tier1 ./my-skill --no-llm

--no-llm disables model-backed stages; it does not remove scanner requirements. Missing required scanner evidence leaves validation INCOMPLETE and exits nonzero. See Tier 1: Validation for the checks and exit codes.

Add deduplication or live evaluation

The live evaluation guide covers runtime readiness and the first with-skill and without-skill comparison. Installation explains how to add each extra to your checkout commands with uv run --python 3.13 --extra EXTRA.