Harness Check
No LLM. No agent run. No API key. In about a second, the harness check predicts whether each agent harness loads your skill, degrades it, or drops it, and what to change.
Results use versioned harness profiles. Add -v to see their verified versions
and dates, or --harness-version HARNESS=VERSION to flag a different release.
See the CLI reference for all options.
A skill can follow the Agent Skills specification and
still never reach the model. Every harness has its own loader, with its own YAML
parser, folders and size limits. The same SKILL.md can work in Claude Code and
be dropped by Codex without an error. The harness check finds those differences
before your users do.
What you get
This example skill has two small problems. The front matter has license twice,
and the body uses $ARGUMENTS.
Read it from the top:
- The first row has one mark per harness, so you see the result at a glance.
- Each line below is one harness: its verdict and its most serious finding.
+1 moremeans it has more findings.1 notecounts notes that don’t change the verdict. A harness with nothing to report gets no line. - The bottom corner is the tally, here 4 of 6 not loaded.
- The line under the panel appears when the same problem also fails a Tier 1
schema check, and points you to
validate --tiers 1.
Each harness gets one of three verdicts:
The compact view labels a finding with its outcome when the rule names one, such
as not found, rejected, listing shortened or delivery truncated.
See every finding and its fix
Add -v for the detailed view. --harness narrows it to the harnesses you care about.
The detailed view adds:
- The rule ID in brackets, such as
CX-STRICT-YAML, so you can search for it or refer to it. - The fix after the arrow.
- A confidence level for each rule.
- The harness release and date each profile was verified against.
- Your description as the model sees it, for each harness that shortens it in its skill list.
Remove the duplicate key, replace $ARGUMENTS with words, and run it again:
Fix what it finds
Every finding comes with its own fix, so recovering is a short loop.
Run it with -v
The detailed view prints the fix for each finding after the arrow. The JSON
output has the same text in the fix field.
These are the findings you are most likely to meet:
Only ship to some harnesses? Narrow the check with --harness codex,claude-code,
so findings for the others stay out of your way.
Two ways to run it
On its own
Inside validate
Use harness-check when this is the only answer you want. It takes a skill
folder, a folder of skills, or a plugin folder. Its findings never change the
exit code.
What it checks
The rules cover everything between your folder and the model’s context.
The full rule list, harness by harness, is in Tier 1: Validation under harness.
Supported harnesses
Where the rules come from
Each harness has a profile that ships with SkillEvaluator. The profiles come from repeated runs of each harness in which the request sent to the model was captured, so a rule describes what the harness did, not what its documentation says.
Harnesses change between releases, so every profile records the release and date
it was verified against, and -v shows them.
--harness-version codex=0.170.0adds a note when you run a release other than the verified one.--harness-profiles DIRuses your own profile YAML files, so you can try newer profiles without waiting for a release.
Match your setup
Some limits depend on how you run the harness. harness-check takes four flags for that:
Without them, the check assumes an unknown context window, a native provider, a model that opens the skill by itself, and an interactive session. See the CLI reference for every flag.
JSON output
--json FILE writes a harness_compatibility block with one entry per skill and harness.
validate puts the same block in its JSON report.
status is loads, degraded or not_loaded. A finding’s effect is one of
those last two, or advisory for a note. Each entry also has a listed field
with the description as that harness lists it.
When it helps
- Before you publish. One skill, six harnesses, about a second. Fix what gets dropped before anyone installs it.
- When a skill never triggers. Check first that the harness loads it and how much of the description reaches the model’s skill list.
- When you move a skill to another harness. A skill written for Claude Code often relies on placeholders and fields that other harnesses ignore.
- When the skills folder grows. Harnesses give the skill list a budget. Past it, descriptions get shortened and skills get left out.
- In CI. Include it in
validateto report loading problems on each change. Findings are advisory by default; an active policy can make them block the run.
Good to know
Does it change pass or fail?
No. Its findings are MEDIUM for not_loaded, LOW for degraded and INFO for
a note. All three sit below the default gate, so the check never fails a run
unless a policy
raises those severities. harness-check exits 0 whatever it finds.
Is there a score?
No. You get a verdict per harness and a tally such as 4 of 6 not loaded. For a
quality score, run skillevaluator quality-check.
Does it replace a live evaluation?
No. It predicts loader behavior from versioned profiles without running the agent. It doesn’t establish which skill reaches a live model or whether the skill helps once it is there. That is what Tier 3: Live Evaluation measures, by running the agent.
Why does it mention my other skills?
A harness lists your skill together with the skills installed next to it, and
that list has a budget. A skill under a harness root such as .claude/skills
is counted with everything in that folder. A catalog line in the panel names
the folder.
When do the install-location rules apply?
Only when the skill sits under a harness root such as .claude/skills or
.agents/skills. A skill in a working folder is not flagged for being in the
wrong place.