InferenceValidate Inference

Understand Provider Validation

View as Markdown

NemoClaw validates the selected provider and model before it creates a sandbox. The exact request depends on the provider API that the agent uses.

Credential Validation

When credential validation fails, the onboarding wizard lets you re-enter the API key, choose another provider, retry, or exit. NemoClaw retries transient upstream failures before it reports a provider failure.

The nvapi- prefix check applies only to NVIDIA_INFERENCE_API_KEY. OpenRouter keys must be non-empty and begin with sk-or-. Other provider keys use provider-aware validation during the retry flow.

Provider Requests

NemoClaw sends a provider-specific request that exercises the API surface intended for the route.

ProviderValidation request
OpenAITries /responses, then /chat/completions.
NVIDIA EndpointsUses /v1/chat/completions and skips /v1/responses.
OpenRouterUses /v1/chat/completions for catalog, model, and smoke validation.
Google GeminiUses the OpenAI-compatible chat-completions path and skips /v1/responses.
Other OpenAI-compatible endpointTries /v1/responses with tool-calling and streaming checks, then falls back to /v1/chat/completions.
Local NVIDIA NIMUses /v1/chat/completions and skips /v1/responses.

For an OpenAI-compatible endpoint, the runtime defaults to /v1/chat/completions even when the Responses probe succeeds. Set NEMOCLAW_PREFERRED_API=openai-responses before onboarding to select /v1/responses only after the probe verifies the required streaming behavior. Set NEMOCLAW_PREFERRED_API=openai-completions to skip the Responses probe and validate Chat Completions only. The Responses streaming check waits up to 5 seconds for response.output_text.delta before onboarding falls back to Chat Completions. Some Chat Completions validation requests require a structured tool call. If such a request reaches the output-token limit after producing only reasoning content, NemoClaw retries once with a larger token budget. This applies to local runtimes such as Ollama and vLLM. Onboarding continues only when the retry returns a structured tool call. If the retry fails or times out, validation stops without another Chat Completions attempt.

The managed Deep Agents runtime keeps use_responses_api = false and uses Chat Completions through https://inference.local/v1. NEMOCLAW_PREFERRED_API does not change that runtime selection.

Anthropic-Compatible Requests

For Hermes and other agents that use only OpenAI-compatible inference, NemoClaw validates /v1/chat/completions for a custom Anthropic selection. This is the API surface that the managed OpenAI frontend uses at runtime.

Compatible Endpoint Probes

Compatible endpoint validation sends a real inference request because many proxies do not expose /models. For an OpenAI-compatible endpoint, a reasoning model that returns only reasoning content can receive a retry with a larger response budget before NemoClaw reports failure. Route, configuration, and authentication failures still fail immediately.

During one onboarding invocation, NemoClaw can reuse one successful Chat Completions validation instead of sending the same immediate host-side request. Reuse requires all these inputs to match:

  • The public endpoint URL.
  • The model ID.
  • The authentication mode.
  • Whether tool calling is required.
  • The validated DNS IP address set.

Trailing endpoint slashes, duplicate IP addresses, and IP address order do not prevent reuse. NemoClaw sends another validation request in any of these cases:

  • Any listed input differs after NemoClaw normalizes the endpoint URL and IP address set, including when the DNS IP address set changes.
  • The endpoint URL contains embedded credentials, a query string, or a fragment.
  • Validation uses custom headers.
  • The endpoint is an operator-trusted private endpoint.

An endpoint that is reachable only through http://host.openshell.internal:<port> cannot receive the host-side API probe. Verify that route from inside the sandbox after onboarding.