Tier 2: Semantic Deduplication

Tier 2 finds redundant content — inside a single skill and across a skill collection — using embedding similarity, with chat-LLM verification for intra-skill candidates. It needs a configured embeddings provider; intra-skill analysis also needs a chat LLM. See Providers & Credentials to set both up, including the fully local recipe when skill content must not leave your machine.

Both checks read local files only. No external vector database or catalog service is required.

Which check do you need

The two checks differ in what they call out to:

Detect redundancy inside one skill

The repository’s exact SPDX-only HTML preamble is excluded from Markdown chunks, so required license metadata cannot create duplicate-content findings. Comments with any additional text or directive remain part of the comparison.

As a skill grows, its directory accumulates repeated content: sections copied between SKILL.md and reference documents, near-identical guidance under different headings, script docstrings that restate the main instructions. That repetition consumes context without adding information.

Not every similarity is redundant. A short overview in SKILL.md and a detailed explanation in references/ are intentional progressive disclosure. Tier 2 therefore runs two stages — embeddings find likely overlap, then a chat LLM classifies its purpose:

$
skillevaluator context-optimization-check ./my-skill

Useful options:

The default intra-skill threshold (0.80) is higher than the inter-skill default because sections from the same skill naturally share more domain vocabulary.

What gets chunked

Valid mapping frontmatter is removed from Markdown before chunking. Malformed, empty, or non-mapping frontmatter remains part of the analyzed text. Every retained chunk keeps its source file and line range, so findings point you to the exact overlapping sections.

Verdicts and what to do about them

Duplicate findings map to severities by confidence:

If chat-LLM analysis fails for a candidate cluster, Tier 2 reports the incomplete analysis as a blocking error rather than treating the content as clean.

Compare skills across a collection

similarity-check recursively discovers skill manifests, embeds each skill’s name and description by default, and reports pairs at or above the selected threshold.

$
skillevaluator similarity-check ./skills

Every match lands in one of five classifications:

The threshold controls which score bands are returned; it does not change the classification boundaries. The default (0.75) reports SIMILAR and stronger matches.

Use a local catalog

For repeated screening — say, every new skill submission against your team’s existing set — save the collection once and query candidates against it.

$
skillevaluator similarity-check ./skills --save-catalog ./skill-catalog.json

$
skillevaluator similarity-check ./candidate-skill --catalog ./skill-catalog.json

Catalogs contain finite embedding vectors, relative skill paths, display names, descriptions, content fingerprints, and provider, model, mode, and endpoint-fingerprint metadata for compatibility checks. They never contain API keys.

The provider, model, endpoint, vector length, and description-versus-full-body mode used for a query must be compatible with the catalog. A missing, malformed, incompatible, non-finite, or oversized catalog is rejected with an actionable error — query with matching settings, or rebuild the catalog with the settings you intend to use.

Catalogs stay local unless you explicitly share them. Because they contain embeddings and other data derived from skill content, review them before sharing like any other generated project artifact.

Inside validate

validate runs the intra-skill deduplication pass by default as part of a full evaluation. When the optional Tier 2 dependencies or an embeddings provider are unavailable, the pass skips gracefully with a warning — a keyless validate still completes. Use --no-tier2 (or --no-dedup) to disable the pass explicitly:

$
skillevaluator validate ./my-skill --no-dedup

All Tier 2 commands accept the standard report options (-r cli,json,html,markdown and -o). HIGH and CRITICAL findings, along with blocking analysis errors, produce a non-zero exit code; MEDIUM and LOW findings are advisory. Exit behavior is identical in every output mode, so these commands work as CI gates — see Gate Your CI.

Defaults and safety limits

Tier 2 treats skill content and catalog files as untrusted input. Before any provider call, it rejects linked roots, linked or escaping files, non-regular files, and paths outside the verified scan root. Explicit limits bound discovered paths, file counts, per-file and total input bytes, content chunks, candidate clusters, catalog size and entry count, vector length, returned matches, and pairwise scalar work. Catalog loading also rejects duplicate JSON keys, unexpected fields, invalid identities, non-finite vectors, and incompatible provider metadata. If a collection exceeds a limit, split it into intentional batches rather than scanning an unbounded directory tree.

What leaves your machine

• A default inter-skill scan sends skill names and descriptions to the configured embeddings provider. With --full-body, the scan sends each discovered SKILL.md in full, including frontmatter, in embedding chunks; supporting files are not sent.
• Intra-skill analysis sends retained content chunks to the configured embeddings provider. Only candidate clusters found by the embedding stage are sent to the configured chat LLM for classification.
• similarity-check never calls a chat LLM.
• Catalog files are read and written locally; SkillEvaluator does not upload them to a catalog service.

Use the fully local provider recipe when skill content must not leave the machine, and review your provider’s data handling terms before sending confidential skill content to a hosted endpoint.

Edge cases

Next steps