GitLab CI/CD | SkillEvaluator Documentation

Bring skill evaluation to GitLab with a reusable CI job. Point it at your skills and choose a quality threshold; merge requests evaluate changed skills without installing scanners or adding evaluation code to your repository. The shared container includes SkillSpector security scanning.

Get started

Copy gitlab-skill-checks.yml from the ready-to-use templates. It includes the pinned shared configuration and scanner image. You do not need a GitLab mirror or a local onboarding CLI.

I prefer to inspect the source template

Browse the GitLab source template and template catalog. Source files contain placeholders; use the filled-in copy when onboarding.

Add the reusable job

Copy gitlab-skill-checks.yml from the template folder using GitHub’s Raw view. For a new pipeline, use its contents as .gitlab-ci.yml. For an existing pipeline, merge its include, variables and skill-evaluation rules into your root file.

Choose your skills and score

Set SKILL_EVAL_PATHS to your skill directory and SKILL_EVAL_MIN_SCORE to the required quality score from 0 to 100 (default 70). For multiple roots, use a YAML multiline string.

Open a merge request

The job evaluates changed skills, shows the result in the pipeline, and retains reports as job artifacts, including when a check fails.

The generated file has this shape; the release fills in the immutable pins:

.gitlab-ci.yml

include:
  - remote: https://raw.githubusercontent.com/NVIDIA/SkillEvaluator/RELEASE_WORKFLOW_SHA/ci/gitlab/skill-evaluation.yml
variables:
  SKILL_EVAL_IMAGE: ghcr.io/nvidia/skillevaluator-tier1@sha256:RELEASE_IMAGE_DIGEST
  SKILL_EVAL_PATHS: "skills"
  SKILL_EVAL_MIN_SCORE: "70"
skill-evaluation:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
    - if: '$CI_PIPELINE_SOURCE == "web"'

A compatible runner must be able to pull the image and run Linux containers. If it requires tags, add them to skill-evaluation. Static scanning needs no model key, but dependency vulnerability checks may contact external services.

See the results

Open the skill-evaluation job and download its skill-evaluator-report artifact. Open index.html for the consolidated view, then select a skill for its detailed HTML report. JSON, Markdown and SARIF are included.

Example report table showing a failed skill with a quality score of 98.2 and a link to its report — Real demo result: the skill scored 98.2 and failed a 100-point threshold. The score and the scan verdict are separate.

The public examples include this report preview and a log excerpt without requiring access to a separate demo project.

Job log

Report artifact

The log identifies the skill, result and quality score. This shortened excerpt is from the real demo run shown above:

Skill result: "skills/glossary-builder" — FAILED
  Quality: 98.2/100, grade "A"
  Check: "QUALITY" — "failed"
    Scanner error: "Quality score 98.2/100 is below the required 100/100 ..."

When does it run?

SKILL_EVAL_SCAN_MODE: auto evaluates changed skills in detached merge-request pipelines and all configured skills on the default branch or a manual pipeline. Set it to all for a full scan each time. Changed-only selection needs the merge-request diff base and full Git history; missing history is an error, not a pass. Merged-results and merge-train pipelines must use all until changed-only selection is supported for them. Shared files outside skill directories do not select dependent skills; use all when those changes require a complete scan.

Fit the job into an existing pipeline

GitLab requires merge-request rules in the root .gitlab-ci.yml, even when a job is included. The generated file provides them. If your project has workflow: rules, ensure they allow merge-request pipelines while preserving your existing policy. The job uses the test stage by default; override its stage if your pipeline does not define it. A failed or incomplete job fails the pipeline; configure your GitLab merge policy if it should also block merging.

Scores, reports and privacy

The threshold applies to the quality score; security, schema and other required checks remain independent gates. The first version runs Tier 1 only. This integration does not host reports or post merge-request comments automatically. Known values detected by credential patterns, NVIDIA API keys, and unquoted KEY=value and Bearer credentials are redacted, including parts of them that a scanner cuts or escapes, but not every secret or transformation is detectable. Raw scanner logs are withheld when a credential is present. Remove and rotate exposed credentials; previous artifacts and Git history are not rewritten.

For GitHub and other CI systems, see Repository Integration.