Tier 2: Semantic Deduplication

Tier 2 finds redundant content — inside a single skill and across a skill collection — using embedding similarity, with chat-LLM verification for intra-skill candidates. It needs a configured embeddings provider; intra-skill analysis also needs a chat LLM. See Providers & Credentials to set both up, including the fully local recipe when skill content must not leave your machine.

Both checks read local files only. No external vector database or catalog service is required.

Run Tier 2

skillevaluator tier2 ./my-skill

This runs intra-skill analysis. To also compare with other skills, supply a saved local JSON catalog:

skillevaluator tier2 ./my-skill --catalog ./skill-catalog.json

Without --catalog, the output explicitly reports that inter-skill comparison did not run; it does not claim the skill is distinct from a catalog it never checked. Use similarity-check ./skills --save-catalog ./skill-catalog.json to build the catalog. Direct Tier 2 takes one skill directory; the expert similarity-check command also supports collection scans.

Direct Tier 2 prints the terminal report and writes JSON and HTML reports to reports/ by default. Use -r cli for terminal-only output, or -o/--output-dir to choose a different report directory.

For NVIDIA Build or OpenAI, the same provider and key configure both chat and embeddings; no separate embedding variables are needed. See Providers & Credentials for other providers.

Which check do you need

The two checks differ in what they call out to:

Detect redundancy inside one skill

The repository’s exact SPDX-only HTML preamble is excluded from Markdown chunks, so required license metadata cannot create duplicate-content findings. Comments with any additional text or directive remain part of the comparison.

As a skill grows, its directory accumulates repeated content: sections copied between SKILL.md and reference documents, near-identical guidance under different headings, script docstrings that restate the main instructions. That repetition consumes context without adding information.

Not every similarity is redundant. A short overview in SKILL.md and a detailed explanation in references/ are intentional progressive disclosure. Tier 2 therefore runs two stages — embeddings find likely overlap, then a chat LLM classifies its purpose:

skillevaluator context-optimization-check ./my-skill

Useful options:

The default intra-skill threshold (0.80) is higher than the inter-skill default because sections from the same skill naturally share more domain vocabulary.

What gets chunked

Valid mapping frontmatter is removed from Markdown before chunking. Malformed, empty, or non-mapping frontmatter remains part of the analyzed text. Every retained chunk keeps its source file and line range, so findings point you to the exact overlapping sections.

Each embedding input is capped at 3,000 characters, including long table rows, Python docstrings, and shell functions. Oversized Markdown sections are packed paragraph by paragraph up to the cap, and paragraphs under 80 characters stay attached to a neighbouring paragraph rather than becoming chunks of their own. Pieces split from one section, docstring, or function do not count as duplicates of each other. An independent copy can still join a duplicate cluster, and the finding then lists the split source once with its merged line range. Python pieces retain the enclosing source range because cleaned docstrings and function signatures are not contiguous text.

Verdicts and what to do about them

Duplicate findings map to severities by confidence:

If chat-LLM analysis fails for a candidate cluster, Tier 2 reports the incomplete analysis as a blocking error rather than treating the content as clean.

The same applies when a skill exceeds a work limit on content that was already collected: the content-chunk or comparison-work limit, or a limit on the LLM review (candidate clusters, members in one cluster, or prompt size per cluster or in total). Tier 2 stops and reports the scan as INCOMPLETE, not as a duplicate-content finding, with an execution error that names the limit; the JSON report records it under work_limit. A policy severity override cannot turn the incomplete scan into a pass. Split the skill or remove repeated material, then rerun.

A model that declines to classify a cluster (a safety refusal, which some models return for security-themed skill content) is reported differently. The refusal is no evidence of duplication, and the same model usually refuses again on a rerun, so Tier 2 records a MEDIUM llm_refused finding that names the overlapping sections and does not block. Review that overlap by hand, or rerun with another chat model (--llm-model or SKILL_EVAL_LLM_MODEL). To make a refusal block validate, raise CONTENT_DEDUP.llm_refused to high in a policy overlay.

Compare skills across a collection

similarity-check recursively discovers skill manifests, embeds each skill’s name and description by default, and reports pairs at or above the selected threshold.

skillevaluator similarity-check ./skills

Every match lands in one of five classifications:

The threshold controls which score bands are returned; it does not change the classification boundaries. The default (0.75) reports SIMILAR and stronger matches.

The entry budget is independent of the 256-file limit inside one skill. A 343-skill collection fits the defaults, including with 2,048-dimensional embeddings. Entry and comparison budgets are separate: a pairwise scan needs n * (n - 1) / 2 * dimensions scalar comparisons, so increasing the entry budget may also require increasing the comparison budget. For example, 500 skills with 2,048-dimensional embeddings need 255,488,000 scalar comparisons:

skillevaluator similarity-check ./skills --max-entries 500 --max-scalar-comparisons 256000000

The entry budget counts selected manifests even when their frontmatter is invalid. It applies before any embedding request and never truncates a scan. Fresh pairwise scans check the comparison budget after the first validated embedding batch (or first full-body vector), before requesting the remaining embeddings or saving a catalog. Python index-only builds remain available for single-target queries that fit the smaller query budget. A failure names the option to adjust. Increasing either budget can increase provider usage or processing time. Neither option changes the separate path, input-byte, catalog-file, vector, or result limits.

Collection discovery allows up to 20,000 paths, independently of the 4,096-path limit inside one skill. Each visited directory and file counts, including files that are not selected as manifests; excluded directories count once, but their contents are pruned. This accommodates all 5,000 supported entries when each skill has a directory and SKILL.md. Collections with more supporting files or deeper layouts may reach the path ceiling before the selected-entry budget.

For a saved catalog query, --max-entries does not reduce the catalog’s existing 5,000-entry capacity. Its comparison work is catalog entries * dimensions, bounded by --max-scalar-comparisons.

Use a local catalog

For repeated screening — say, every new skill submission against your team’s existing set — save the collection once and query candidates against it.

skillevaluator similarity-check ./skills --save-catalog ./skill-catalog.json

skillevaluator similarity-check ./candidate-skill --catalog ./skill-catalog.json

Re-screening a skill that is already in the catalog does not report it as a duplicate of its own entry. A catalog does not record the directory it was built from, so an entry is skipped as the candidate’s own only when all of these hold:

• Its content fingerprint matches what the query compares: the name and description, or the whole SKILL.md with --full-body.
• Its path ends the candidate’s path — alpha for ./skills/alpha in a catalog built from ./skills — which locates the catalog’s tree around the candidate.
• Of up to eight other catalog skills, spread evenly across the catalog’s path-sorted list, more than half are present unchanged at their recorded paths in that tree, without symlinks. Skills that contain the candidate or are nested inside it are not checked, because a copied folder carries them too.

At most one entry is skipped, and the result records it under excluded_catalog_paths. A catalog with no other skill entry never skips one. A verbatim copy elsewhere — in a staging folder, another repository, or a new folder of the same repository — is still compared and reported, unless it sits in a copy of most of the catalog’s tree, such as a second checkout of the same repository, which counts as the same tree. An edited skill whose compared content changed no longer matches its old entry and is compared with it like any other skill; to screen edited skills, build the catalog from the same tree you screen.

Catalogs contain finite embedding vectors, relative skill paths, display names, descriptions, content fingerprints, and provider, model, mode, and endpoint-fingerprint metadata for compatibility checks. They never contain API keys.

The provider, model, endpoint, vector length, and description-versus-full-body mode used for a query must be compatible with the catalog. A missing, malformed, incompatible, non-finite, or oversized catalog is rejected with an actionable error — query with matching settings, or rebuild the catalog with the settings you intend to use.

Catalogs stay local unless you explicitly share them. Because they contain embeddings and other data derived from skill content, review them before sharing like any other generated project artifact.

Inside validate

validate runs the intra-skill deduplication pass by default as part of a full evaluation. When the optional Tier 2 dependencies or an embeddings provider are unavailable, the pass skips gracefully with a warning. The full skill pipeline also includes Tier 3, which needs provider access and a sandbox. Use --no-tier2 (or --no-dedup) to disable only the deduplication pass:

skillevaluator validate ./my-skill --no-dedup

This still runs Tier 3 with autopilot. For keyless static checks only, select skillevaluator validate ./my-skill --tiers 1.

All Tier 2 commands accept the standard report options (-r cli,json,html,markdown and -o). HIGH and CRITICAL findings, along with blocking analysis errors, produce a non-zero exit code; MEDIUM and LOW findings are advisory. Exit behavior is identical in every output mode, so these commands work as CI gates — see Gate Your CI.

Inside the combined validate pipeline, Tier 2 also gates by default. Pass --no-block-on-dedup to keep the scan and its findings in every report without letting those findings change validate’s exit code. Use --no-dedup only when you intend to skip Tier 2 entirely.

Defaults and safety limits

Intra-skill clustering compares every pair of chunks, so its work is n * (n - 1) / 2 * dimensions. A skill over the chunk ceiling stops before any embedding request. A skill within the ceiling whose embeddings are too wide for its chunk count stops after the first embedding batch, which sets the width. Either way the scan is reported as INCOMPLETE. For the comparison-work limit (scalar_comparison_limit), the execution error and the JSON work_limit record name the effective chunk limit for that embedding width (effective_chunk_limit) and the skill’s chunk count (chunk_count). Split the skill, remove repeated material, or use a narrower embedding model.

Tier 2 treats skill content and catalog files as untrusted input. Before any provider call, it rejects linked roots, linked directories and manifests, hard-linked selected files, non-regular files, paths outside the verified scan root, and all file redirects except the exact contained CLAUDE.md -> AGENTS.md compatibility alias. That alias is validated but never followed; its exactly named, independently discovered, single-link regular sibling target is scanned once. Explicit limits bound discovered paths, file counts, per-file and total input bytes, content chunks, candidate clusters, catalog size and entry count, vector length, returned matches, and pairwise scalar work. Catalog loading also rejects duplicate JSON keys, unexpected fields, invalid identities, non-finite vectors, and incompatible provider metadata. Increase the documented collection budgets only for a workload you intend to process. Splitting a collection into independent batches omits comparisons between batches and is not a complete deduplication scan.

What leaves your machine

• A default inter-skill scan sends skill names and descriptions to the configured embeddings provider. With --full-body, the scan sends each discovered SKILL.md in full, including frontmatter, in embedding chunks; supporting files are not sent.
• Intra-skill analysis sends retained content chunks to the configured embeddings provider. Only candidate clusters found by the embedding stage are sent to the configured chat LLM for classification.
• similarity-check never calls a chat LLM.
• Catalog files are read and written locally; SkillEvaluator does not upload them to a catalog service.

Use the fully local provider recipe when skill content must not leave the machine, and review your provider’s data handling terms before sending confidential skill content to a hosted endpoint.

Edge cases

Next steps