Environment Variables
Every environment variable SkillEvaluator reads, grouped by feature area. Each group links to the guide that explains the workflow the variables belong to — this page is the inventory, not the tutorial.
Boolean variables accept 1, true, yes, or on (case-insensitive) as true, and 0, false, no, or off as false.
Two historical SkillEvaluator prefixes coexist. SKILL_EVAL_* covers provider
and model configuration plus a small set of Tier 3 engine knobs;
SKILLEVALUATOR_* covers product-level validation, report paths, scanner
paths, and sandbox/runtime behavior. The prefixes are part of each variable’s
literal public name and are not interchangeable. The separately documented
SKILLSPECTOR_* family belongs to SkillSpector, provider credential families
belong to their providers, and
Harbor, the open-source agent
evaluation framework used for Tier 3, owns the HARBOR_* in-container grader
contract. None belongs to either SkillEvaluator prefix.
LLM provider
These variables select and configure the provider used for LLM-backed checks and judging. The setup walkthrough, including per-provider export blocks and default models, is in Providers & Credentials.
Anthropic endpoint roots
When anthropic is the selected evaluator provider, endpoint precedence is SKILL_EVAL_LLM_BASE_URL, then ANTHROPIC_BASE_URL, then the Anthropic default. A root such as https://gateway.example.com and a legacy terminal /v1 root such as https://gateway.example.com/team/v1 both produce request paths with exactly one /v1/messages suffix for the Anthropic SDK, the Tier 3 verifier, and a matching Claude Code agent. An independently credentialed Claude Code route applies the same normalization and validation to ANTHROPIC_BASE_URL when the evaluator uses another provider. Routing through path prefixes, ports, IP addresses, internationalized domain names, and ordinary safe percent escapes is preserved.
Anthropic configuration fails before Harbor starts for a full /v1/messages endpoint, embedded credentials, a query or fragment, whitespace or control characters, a backslash, an invalid authority, interior empty or dot path segments, or ambiguous multi-layer endpoint encoding. This normalization applies only to Anthropic. OpenAI-compatible endpoints commonly include /v1, which remains unchanged.
Auto-selection
When SKILL_EVAL_LLM_PROVIDER is unset, auto-selection only happens when exactly one public credential is exported:
NVIDIA_API_KEY→nv_buildOPENAI_API_KEY→openaiANTHROPIC_API_KEY→anthropic
If none is set, LLM-backed commands fail with a configuration error. If more than one is set, that is also a configuration error — there is no silent preference order, so set SKILL_EVAL_LLM_PROVIDER explicitly when multiple keys are exported.
SkillSpector bridge
The --llm security analysis runs SkillSpector, which has its own provider environment. SkillEvaluator bridges your configured provider into that invocation with an isolated subprocess environment: only the selected provider’s settings and a small set of basic process variables are forwarded, so ambient credentials for other providers never reach SkillSpector. The nv_build provider is bridged through SkillSpector’s OpenAI-compatible path, pointing OPENAI_BASE_URL at the public NVIDIA Build endpoint.
Embeddings
Tier 2 semantic checks use an OpenAI-compatible embeddings API. Workflow context lives in Providers & Credentials and Tier 2: Deduplication.
Validation (Tier 1)
These variables tune the static and security checks. The check-by-check guide is Tier 1: Validation.
The scanner path overrides fail closed: a value that is not an absolute, executable file is treated as a configuration error rather than silently falling back to another same-named binary on PATH. This prevents a planted executable from hijacking a security scan.
Live evaluation (Tier 3)
These variables control where results land and how Harbor confines local mode. The run workflow is Tier 3: Live Evaluation; execution environments are covered in Agents & Sandboxes.
Tier 3 judge failure semantics
In standard grading, a required judge transport, authentication, timeout, HTTP, or exhausted response-parsing error is an evaluation infrastructure failure, not a numeric 0.0. A condition containing a failed or incomplete default trial publishes none of its scores, dimensions, pass@k, or quality findings, and no cross-condition lift is published. Redacted diagnostics and coverage information remain available. A genuine model score of 0.0 remains valid and is published normally.
default_plus_custom cannot use a custom score to rescue failed standard judging. custom_only is unchanged and does not invoke the standard judge.
In-container grader contract
The HARBOR_* variables are read by your custom grader inside the task container — you never set them yourself, and each has a fixed in-container default. The full reward contract, with defaults and examples, lives in Custom Graders & Tasks.
Set at runtime, not committed
Keep credentials out of skill repositories and CI configuration files. Export provider keys as masked secrets at runtime. Agent credentials for Tier 3 come only from the host environment — evals/config.yml runtime_env carries non-credential values, and any entry that names or references an operator-owned credential (OPENAI_API_KEY, ANTHROPIC_API_KEY, NVIDIA_API_KEY, their base-URL variables, CLAUDE_CODE_USE_BEDROCK, and the AWS_* credential variables) is rejected at run time, as are names that control the host process (PATH, DOCKER_*, proxy variables, SKILLEVALUATOR_*, and similar). See the blocked-names table in Providers & Credentials. With NVIDIA Build, one exported NVIDIA_API_KEY covers both the evaluator and the agents in the docker and local env-modes.
See also
- Providers & Credentials — the setup walkthrough behind the provider variables
- CLI Reference — the per-run flags that override many of these variables
- Gate Your CI — supplying these variables as masked CI secrets