> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/skills/skillevaluator/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/skills/skillevaluator/_mcp/server. # Providers & Credentials > Configure LLM providers, embeddings, and agent credentials for SkillEvaluator — and see exactly which commands run with no key at all. This page gets SkillEvaluator's LLM-backed features working: choosing a provider, wiring embeddings for Tier 2, and separating evaluator credentials from agent credentials in Tier 3. Most of Tier 1 needs no credential at all — start with the map below to see whether you need a key in the first place. For the exhaustive variable-by-variable inventory, see [Environment Variables](/skills/skillevaluator/environment-variables). Tier 3 executes trials through [Harbor](https://github.com/harbor-framework/harbor), the open-source agent evaluation framework. This page separates the credentials SkillEvaluator and the live agents need from the backend configuration Harbor itself reads. ## What needs a key Tiers describe evaluation depth; credentials are a separate axis. This is the complete map — everything not listed under "needs a provider" runs fully offline: | Runs with no credential at all | Needs a provider key | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `validate --no-dedup` (all deterministic checks: schema, security scanners including secret detection, PII, license, code-integrity, unicode, quality, lint) | `rubric-eval` — LLM-as-judge scoring (chat LLM) | | `quality-check`, `security-scan`, `pii-scan`, `lint-scripts` | `validate --llm` / `security-scan --llm` — deeper LLM security analysis | | `doctor`, `health-check` (offline readiness probes; missing credentials are reported as not ready) | `--llm-verify` on `validate`, `security-scan`, `pii-scan` — LLM false-positive suppression | | | All Tier 2 commands — embeddings API; `context-optimization-check` and `dedup-scan` also use a chat LLM (`similarity-check` is embeddings-only). A local OpenAI-compatible server works too — see [Fully local setup](#fully-local-setup) | | Tier 3 scaffolding and inspection: `create-eval-dataset --no-llm`, `init-custom-grader`, `init-harbor-task`, `tier3 validate`, `view`, `harbor-view`, and `compare` | Tier 3 LLM dataset generation needs a provider key; `tier3 evaluate` needs a provider key plus a credential the selected live agent can use — with NVIDIA Build in `--env-mode docker` or `local`, one `NVIDIA_API_KEY` covers both roles | | | `models` — lists the selected provider's authenticated model catalog | `validate` without `--no-dedup` stays usable keyless: the Tier 2 dedup pass skips gracefully when no embedding provider is configured. ## Choose an LLM provider Set `SKILL_EVAL_LLM_PROVIDER` to one of `openai`, `anthropic`, `nv_build`, `bedrock`, or `openai-compatible`. The selected provider determines which credential SkillEvaluator reads. All but `openai-compatible` ship a default model that `SKILL_EVAL_LLM_MODEL` overrides. #### OpenAI ```bash title="OpenAI setup" export SKILL_EVAL_LLM_PROVIDER=openai export OPENAI_API_KEY='sk-...' ``` Default model: `gpt-5.4-mini`, endpoint `https://api.openai.com/v1`. Override the endpoint with `SKILL_EVAL_LLM_BASE_URL` (takes precedence) or `OPENAI_BASE_URL` — but a base URL that is a non-canonical alias of the native OpenAI endpoint is rejected; use `https://api.openai.com/v1` verbatim. On the canonical endpoint, `gpt-5*` models automatically send `max_completion_tokens` instead of `max_tokens`. #### Anthropic ```bash title="Anthropic setup" export SKILL_EVAL_LLM_PROVIDER=anthropic export ANTHROPIC_API_KEY='sk-ant-...' ``` Default model: `claude-sonnet-4-5`, using the Anthropic SDK's default endpoint. Override with `SKILL_EVAL_LLM_BASE_URL` (takes precedence) or `ANTHROPIC_BASE_URL`. #### NVIDIA Build ```bash title="NVIDIA Build setup" export SKILL_EVAL_LLM_PROVIDER=nv_build export NVIDIA_API_KEY='nvapi-...' ``` Default model: `nvidia/nemotron-3-nano-30b-a3b`. The endpoint is fixed to `https://integrate.api.nvidia.com/v1` — `SKILL_EVAL_LLM_BASE_URL` and `SKILL_EVAL_EMBEDDING_BASE_URL` do not override it. To point at a custom or self-hosted endpoint, use the `openai-compatible` provider instead. The quickest start is a free NVIDIA API Catalog key from [build.nvidia.com](https://build.nvidia.com/), exported as `NVIDIA_API_KEY` — it covers LLM judging and Tier 2 embeddings with one credential. #### Bedrock ```bash title="Amazon Bedrock setup" export SKILL_EVAL_LLM_PROVIDER=bedrock export AWS_REGION=us-west-2 # default when unset # Authenticate with the standard AWS credential chain # (environment variables, shared profile, or an assumed role). ``` Default model: `us.anthropic.claude-sonnet-4-5-20250929-v1:0` on the AWS Bedrock Runtime. #### OpenAI-compatible ```bash title="OpenAI-compatible setup" export SKILL_EVAL_LLM_PROVIDER=openai-compatible export SKILL_EVAL_LLM_BASE_URL='http://your-server:8000/v1' export SKILL_EVAL_LLM_MODEL='your-model-name' export SKILL_EVAL_LLM_API_KEY='your-key' ``` All three values are required — there is no default model or endpoint. Any OpenAI-compatible server works: vLLM, Ollama, llama.cpp, NVIDIA NIM, or a hosted gateway. ### Auto-selection When `SKILL_EVAL_LLM_PROVIDER` is unset and exactly one of these credentials is present, SkillEvaluator selects the matching provider: * `NVIDIA_API_KEY` → `nv_build` * `OPENAI_API_KEY` → `openai` * `ANTHROPIC_API_KEY` → `anthropic` If none of those variables is set, LLM-backed commands fail with a configuration error. If more than one is exported, resolution also fails — `SKILL_EVAL_LLM_PROVIDER` is required when multiple public provider credentials are configured; there is no silent preference order. ## Embeddings Tier 2 uses an OpenAI-compatible embeddings API. `SKILL_EVAL_EMBEDDING_PROVIDER` selects it; when unset, it falls back to your `SKILL_EVAL_LLM_PROVIDER` (or the same auto-selection order as above). | Provider (`SKILL_EVAL_EMBEDDING_PROVIDER`) | Credential | Default embedding model | | ------------------------------------------ | ----------------------------------------------------------------------- | ---------------------------------------------------- | | `openai` | `OPENAI_API_KEY` | `text-embedding-3-small` | | `nv_build` | `NVIDIA_API_KEY` | `nvidia/nv-embed-v1` | | `openai-compatible` | `SKILL_EVAL_EMBEDDING_API_KEY` (falls back to `SKILL_EVAL_LLM_API_KEY`) | *(explicit — `SKILL_EVAL_EMBEDDING_MODEL` required)* | Use `SKILL_EVAL_EMBEDDING_MODEL` and `SKILL_EVAL_EMBEDDING_BASE_URL` to override defaults; for `openai-compatible`, the base URL falls back to `SKILL_EVAL_LLM_BASE_URL`. The `nv_build` endpoint is fixed to `https://integrate.api.nvidia.com/v1` — only its model can be overridden. Anthropic and Bedrock do not provide embeddings, and SkillEvaluator rejects them as embedding providers. When one of those is your LLM provider, configure an embedding-capable provider separately for Tier 2: ```bash title="Separate embedding provider" export SKILL_EVAL_EMBEDDING_PROVIDER=openai export OPENAI_API_KEY='sk-...' ``` ## Fully local setup The `openai-compatible` provider accepts any local OpenAI-compatible server (Ollama, vLLM, llama.cpp, NVIDIA NIM), so Tier 2 can run with no external calls at all. Example recipe using Ollama in Docker: ### Start a local model server ```bash title="Ollama in Docker" docker run -d --name ollama -p 11434:11434 ollama/ollama docker exec ollama ollama pull nomic-embed-text # embeddings docker exec ollama ollama pull qwen2.5:0.5b # chat LLM for dedup analysis ``` ### Point SkillEvaluator at it ```bash title="Local provider configuration" export SKILL_EVAL_EMBEDDING_PROVIDER=openai-compatible export SKILL_EVAL_EMBEDDING_BASE_URL=http://localhost:11434/v1 export SKILL_EVAL_EMBEDDING_MODEL=nomic-embed-text export SKILL_EVAL_EMBEDDING_API_KEY=local-no-key # must be set; local servers ignore the value # context-optimization-check and dedup-scan also use a chat LLM: export SKILL_EVAL_LLM_PROVIDER=openai-compatible export SKILL_EVAL_LLM_BASE_URL=http://localhost:11434/v1 export SKILL_EVAL_LLM_MODEL=qwen2.5:0.5b export SKILL_EVAL_LLM_API_KEY=local-no-key ``` ### Run the Tier 2 checks ```bash title="Fully local Tier 2" skillevaluator similarity-check ./skills skillevaluator context-optimization-check ./my-skill ``` `similarity-check` compares skills directly or through a local catalog and needs only the embedding variables. `context-optimization-check` (also available as its `dedup-scan` alias) additionally uses the chat model. Analysis quality tracks that model — the tiny model above proves the plumbing; pick a stronger local model for verdicts you intend to act on. ## Tier 3 credentials: two roles Live evaluation involves two distinct credentials, and they are not interchangeable: | Role | Who uses it | How you configure it | | ---------------- | ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | Evaluator key | SkillEvaluator itself — dataset generation and verifier-side judging | The provider setup above (`SKILL_EVAL_LLM_PROVIDER` + its key) | | Agent credential | The live agent under test (Codex, Claude Code, ...) inside its task environment | Your host environment only — export the agent's native variables (for example `OPENAI_API_KEY` and `OPENAI_BASE_URL` for Codex) before the run | Both roles resolve from the operator's environment; a skill's `evals/config.yml` can never supply, rename, or redirect a credential. What each agent needs depends on the provider and `--env-mode`: * **NVIDIA Build, one key** (`--env-mode docker` or `local`): a single `NVIDIA_API_KEY` powers the evaluator and all three supported agents — `opencode` calls NVIDIA Build directly, while `claude-code` and `codex` route through SkillEvaluator's compatibility bridge (in-container in Docker mode; an authenticated in-process host bridge in `local` mode). Agent models must be full NVIDIA Build catalog IDs in `publisher/model` form. * **NVIDIA Build on Harbor cloud modes** (`daytona`, `e2b`, ...): the bridge is not available. `claude-code` needs its own native `ANTHROPIC_API_KEY`, `codex` needs a full OpenAI Responses API credential (`OPENAI_API_KEY` plus `OPENAI_BASE_URL`), and each needs an explicit model via `--agent-model` or `harbor.agents..model`. * **Matching native provider**: when the evaluator provider is the agent's own (for example `anthropic` with `claude-code`, or `openai` with `codex`), the same key serves both roles automatically. The agent's model lives in the skill's `evals/config.yml`: ```yaml title="evals/config.yml" schema_version: 1 harbor: agents: codex: model: gpt-5.4-mini ``` You can also supply that model at invocation time with `--agent-model codex=MODEL`. See [Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation) for the full run workflow. In Docker mode, credential values are never rendered on the `docker compose exec` command line — the bridge receives them through a mode-0600 file inside the container that is consumed and removed before the agent starts. ### List the models your key can access Before pinning `--agent-model` or `SKILL_EVAL_LLM_MODEL`, check what the selected provider actually serves: ```bash title="List provider models" skillevaluator models --limit 20 skillevaluator models --json ``` `skillevaluator models` prints a filtered, bounded view of the authenticated provider catalog (`--limit` accepts 1–100, default 10; `--json` emits machine-readable output). It uses the same provider and credential resolution as evaluation, so it doubles as a quick check that your key works. A model appearing in the catalog is not proof a given agent harness supports it — use `doctor --verify-models` for that. Bedrock exposes no HTTP model catalog, so `models` errors on that provider; use `doctor --verify-models` there too. ### `runtime_env` is for task values, not credentials `runtime_env` in `evals/config.yml` remains valid for non-credential task values — plain names the task environment needs. Credential names are rejected outright: * Any operator-owned credential name (`OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, `NVIDIA_API_KEY`, `CLAUDE_CODE_USE_BEDROCK`) fails the run with `harbor.runtime_env contains operator-owned credential name(s)`. * A `${NVIDIA_API_KEY}`-style reference to an operator-owned credential inside any value is rejected too, so a credential cannot be aliased under a different name. ### Blocked `runtime_env` names Names that control the launcher, credentials, or dynamic runtime are rejected: | Blocked group | Examples | | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Operator-owned agent credentials | `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, `NVIDIA_API_KEY`, `CLAUDE_CODE_USE_BEDROCK` — as names or as `${...}` references in values | | Host path and loader control | `PATH`, `PATHEXT`, `LD_*`, `DYLD_*`, `PYTHON*` | | Container and backend selection | `DOCKER_*`, `COMPOSE_*`, `HARBOR_*`, plus each Harbor backend's own credential variables | | AWS credential chain | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`, `AWS_PROFILE`, `AWS_ROLE_ARN`, and the rest of the `AWS_*` credential-chain variables | | Network and TLS control | `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`, `ALL_PROXY`, `SSL_CERT_FILE`, `SSL_CERT_DIR`, `REQUESTS_CA_BUNDLE`, `SSLKEYLOGFILE`, `SSH_AUTH_SOCK` | | Tracing and tooling | `OTEL_*`, `SKILL_EVAL_*`, `SKILLEVALUATOR_*`, `GIT_*`, `NODE_*`, `PIP_*`, `UV_*` | Configure the selected Harbor backend in the host environment instead. ## SkillSpector bridge The `--llm` security analysis runs [SkillSpector](https://github.com/NVIDIA/SkillSpector), which has its own provider environment (`SKILLSPECTOR_PROVIDER` plus per-provider credential variables). SkillEvaluator bridges your configured provider automatically for that invocation, so the one key above is all you need. For `nv_build`, the bridge uses SkillSpector's OpenAI-compatible provider path pointed at the public Build endpoint, so the same `NVIDIA_API_KEY` covers this too; it does not create a second NVIDIA credential name. The subprocess environment is isolated. Only the selected provider settings and basic process environment variables (`PATH`, `HOME`, TLS bundles, temp dirs) pass through — other ambient credentials in your shell are never forwarded. Setting `SKILLSPECTOR_PROVIDER` yourself overrides the bridge, but it must name a supported public provider (`openai`, `anthropic`, or `bedrock` — or `nv_build` to force the Build mapping); anything else fails closed and the subprocess receives no credential. ## Troubleshooting #### "No provider is configured (SKILL\_EVAL\_LLM\_PROVIDER unset and no credential found)" No provider is selected and no known key is present. Export one of `NVIDIA_API_KEY`, `OPENAI_API_KEY`, or `ANTHROPIC_API_KEY`, or set `SKILL_EVAL_LLM_PROVIDER` explicitly alongside its credential. #### "SKILL\_EVAL\_LLM\_PROVIDER is required when multiple public provider credentials are configured" More than one of `NVIDIA_API_KEY`, `OPENAI_API_KEY`, and `ANTHROPIC_API_KEY` is exported, so auto-selection refuses to guess. Set `SKILL_EVAL_LLM_PROVIDER` to the provider you want (single-key auto-detection is unaffected). Relatedly, an exported-but-empty `SKILL_EVAL_LLM_MODEL` is also an error — unset it to use the provider default, or set a model name. #### "SKILL\_EVAL\_EMBEDDING\_PROVIDER is required because anthropic does not provide embeddings" Your LLM provider is `anthropic` or `bedrock`, and neither offers an embeddings API. Set `SKILL_EVAL_EMBEDDING_PROVIDER` to `openai`, `nv_build`, or `openai-compatible` with the matching credential — see [Embeddings](#embeddings). #### "SKILL\_EVAL\_LLM\_MODEL is required for openai-compatible providers" The `openai-compatible` provider has no defaults. Set all three of `SKILL_EVAL_LLM_MODEL`, `SKILL_EVAL_LLM_BASE_URL`, and `SKILL_EVAL_LLM_API_KEY` (local servers still require the key variable to be set, even if they ignore its value). ## Next steps #### [Tier 2: Deduplication](/skills/skillevaluator/tier2-deduplication) Put your embedding provider to work — find overlap within and across skills. #### [Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation) Run a live agent evaluation with the two-credential setup above.