Tier 1: Static and Security Validation

A custom policy overlays the external profile. It can narrow the accepted author domain and change severities by CATEGORY.check_name or CATEGORY.*:

1
profile: community-strict
2
identity:
3
  author_email_regex: '<[^>]+@example\.org>'
4
severity_overrides:
5
  SCHEMA.author_missing: critical
6
  LICENSE.*: critical

Validates SKILL.md frontmatter and repository placement:

• YAML frontmatter must parse and satisfy the skill schema.
• name and description are required. Names are 1–64 characters; descriptions are 1–1024 characters.
• Names use kebab-case, start with a letter, and cannot have trailing or consecutive hyphens. The directory name must match the frontmatter name. The reserved words anthropic and claude must not appear in a skill name.
• alwaysApply and globs are forbidden skill fields. Optional fields include license, compatibility, metadata, and allowed-tools.
• The standard hierarchy is skills/<skill-name>/ or team-skills/<team>/<skill-name>/. A nonstandard standalone location is reported as an advisory finding.
• Recognized skill-root subdirectories are agents/, references/, scripts/, assets/, evals/, tests/, tools/, and config/; anything else gets a LOW advisory finding. Extend the set per repository with SKILLEVALUATOR_SCHEMA_ALLOWED_DIRS (comma-separated names are added to the defaults, never replace them).
• SKILL.md should stay at or below 500 lines; larger files receive an advisory finding.
• The default profile expects metadata.author in Name <email@host> form but does not require a particular email domain.

To fix common findings: rename the directory to match name, remove forbidden fields, and move the skill under skills/.

security-scan, and the security check inside validate, run SkillSpector. Static scanning covers common skill attack patterns such as prompt injection and data exfiltration; each finding carries the matched pattern, severity, and location.

$
skillevaluator security-scan ./my-skill
$
skillevaluator security-scan ./my-skill --llm
$
skillevaluator security-scan ./my-skill --llm --llm-verify

The LLM pass requires a configured provider. It enriches the static result but does not replace static findings or make missing scanner evidence pass.

pii-scan, and the pii check inside validate, use configurable regex patterns to detect sensitive values.

Common placeholders such as YOUR_API_KEY_HERE, example, test, dummy, and placeholder, along with @example.com and @test.com, are excluded, as are configured allowed paths.

The home-path check compares detected user paths against the submitter’s identity. When no identity is available, that check is skipped and logged — set SKILLEVALUATOR_SUBMITTER (or add author to SKILL.md) to enable it. SKILLEVALUATOR_SUBMITTER is combined with GITHUB_ACTOR, USER, LOGNAME, USERNAME, the OS login name, and the skill’s declared author — every identity that resolves is protected.

Looks for license evidence in frontmatter, common license files, and SPDX headers. Detected identifiers are normalized and matched against the packaged permissive allowlist and restrictive blocklist. Missing or unknown licenses are reported for review; blocked licenses fail according to the active profile (CRITICAL under the default external profile).

Combines three validation groups:

• Code risk (Bandit and Semgrep) — SQL or command injection, hardcoded passwords, insecure cryptography, unsafe deserialization, shell=True, eval()/exec(), CWE patterns, and language-specific anti-patterns.
• Secrets (Gitleaks) — API keys, tokens, passwords, private keys, database credentials, and other high-confidence secret patterns.
• Hygiene — dead relative Markdown links, banned or unpinned dependency declarations, and static discovery of conventional Python test filenames.

Default Tier 1 does not import or execute target-controlled Python code. The test_discovery result is filename evidence only: it counts regular, in-tree, non-symlink test_*.py and *_test.py candidates without parsing or importing them. It reports execution_performed=false and coverage_measured=false; test success and code coverage must be established separately in a trusted project environment or an explicit sandbox.

A missing, timed-out, crashed, or malformed Bandit, Semgrep, Gitleaks, or SkillSpector run is treated as INCOMPLETE: terminal and saved reports are non-green and validate exits non-zero. See Scanner setup for installation.

Detects invisible Unicode characters associated with ASCII smuggling, hidden-data encoding, and trojan-source attacks.

Severity increases with suspicious run length. A BOM (U+FEFF) at byte zero is downgraded to INFO, and binary files are skipped using MIME and null-byte detection.

quality-check, and the quality check inside validate, produce a 0–100 composite score across four weighted categories.

Grades are A for scores of at least 90, B for at least 80, C for at least 70, D for at least 60, and F below 60. The default --min-score is 70.

The checker detects five skill shapes and applies relevant checks: script-based (Python or shell files under scripts/), lib-based (a Python module containing __init__.py), resource-based (assets, templates, design-system content, or resources), guide-only (documentation without executable or resource content), and hybrid (scripts plus library or resource content).

rubric-eval sends bounded skill documentation and selected supplementary content to your configured LLM provider. It scores nine criteria:

Each criterion is scored from 0 to 10. The local evaluator, not the model’s boolean, treats 7 or higher as passing. The overall score is an importance-weighted mean, and the rubric passes only when that score reaches --min-score and every criterion passes.

$
skillevaluator rubric-eval ./my-skill --min-score 70

lint-scripts, and the lint check inside validate, parse Python scripts under scripts/ and report advisory findings. These findings do not make the Tier 1 gate fail.

The default version check validates an optional metadata.version as a three-part numeric value in major.minor.patch form:

1
metadata:
2
  version: "1.2.3"

When --previous-version or SKILLEVALUATOR_PREVIOUS_VERSION is set, the current version must be present and numerically greater than the previous value. Without a previous-version bound, the version remains optional and the skill can rely on commit history. A previous-version bound is valid only when checking one skill; catalog validation must check each skill separately with its own bound.

$
skillevaluator validate ./my-skill --previous-version 1.2.2

Scans requirements*.txt for known Python package vulnerabilities. When pyproject.toml is present, pip-audit --local audits the active Python environment rather than resolving the dependencies declared in that file. pip-audit is the primary scanner; Safety supplies secondary coverage for requirements files when installed separately. The audit may use network-backed vulnerability databases.

$
skillevaluator validate ./my-skill --checks dependency

Install it with brew install gitleaks or download a binary from the official Gitleaks releases.

Install Bandit and pip-audit with skillevaluator[security]. Install Semgrep separately with brew install semgrep or uv tool install semgrep, and SkillSpector with uv tool install git+https://github.com/NVIDIA/SkillSpector.git. Required scanner failures remain INCOMPLETE until the scanner is installed and returns valid evidence.

Configure a supported provider or omit --llm/--llm-verify. Use --no-tier2 (or --no-dedup) to disable dedup when a fully key-free Tier 1 run is required.

Confirm that the skill contains supported text or code extensions such as .py, .sh, .yaml, .yml, .json, .md, or .txt.

Use an existing, executable absolute path for the relevant SKILLEVALUATOR_*_PATH variable.