InferenceValidate Inference

Verify the Sandbox Inference Route

View as Markdown

Verify inference through the same provider path that the agent uses inside the sandbox. Reading the active route confirms configuration, but it does not authenticate a model request.

Confirm the Configured Route

Read the selected inference path first.

nemohermes inference get

For a selected sandbox with a valid native NVIDIA attachment, the command returns that sandbox’s recorded provider, model, and https://integrate.api.nvidia.com/v1 endpoint. For other providers, the command reads the live shared gateway route. Confirm that the provider, model, and endpoint match the path you intended to configure.

Check Sandbox Inference Health

Run the named sandbox status command.

nemohermes <name> status

For NVIDIA Endpoints, the Inference row verifies the recorded sandbox attachment. It then sends a native Chat Completions request to https://integrate.api.nvidia.com/v1. It does not probe https://inference.local/v1/models for this provider.

For shared-route providers, the Inference row first checks the sandbox’s inference.local path. When that route responds, status sends an inference request through the same path. When the live provider matches the recorded provider, status validates the result against the recorded API family, even when only the model differs. When the live provider differs, status does not carry the recorded API family to the live provider. The row reports healthy only when the route returns a structurally valid result for the selected API family. An empty body, malformed JSON, provider-error envelope, or wrong response shape reports unhealthy, even with a 2xx status. Status diagnostics do not include the response body. An ordinary HTTP 404 from /v1/models reports an unhealthy route because the model catalog is absent. The recorded openrouter-api provider uses NemoClaw’s Chat Completions-only OpenRouter adapter. Its models route returns 404 by design, so status requires a successful inference request before it reports the route as healthy. When the models route responds and status sends an inference request, a failing row names the endpoint that request used. When the models route itself does not respond, the row names the models route. An HTTP 401 or 403 response reports unauthorized. Correct the stored provider credential. An HTTP 404 for a model that is in the NVIDIA Build catalog but is not deployed for your account explains that cause and prompts you to select a different model. The route-reachability and upstream provider subprobes remain available to identify the failing hop. The route-reachability subprobe accompanies a row that sent an inference request. It reads reachable for a 2xx models route and reports the status the models route returned for any other answer. The provider, model, and endpoint appear with the rest of the sandbox state. This path includes the OpenShell proxy and its authentication rewrite. When onboarding prints a dashboard summary, use it to verify that NemoClaw ran the same route-reachability probe from inside the sandbox. Treat an unreachable route, an unsupported models-route HTTP 404, or HTTP 5xx as a failed readiness check: onboarding marks the sandbox not ready and exits non-zero. Restore the configured endpoint or proxy, run nemohermes onboard --resume to complete the retained onboarding session, then rerun the status command.

Understand Local Provider Post-Ready Checks

For local Ollama, local vLLM, and local NVIDIA NIM on Docker GPU sandboxes using the compatibility route, onboarding performs an additional check after the sandbox becomes ready. Local NIM uses the vllm-local route, so it receives the same reversible post-ready check as local vLLM. It requests https://inference.local/v1/models from inside the sandbox and accepts only a 2xx response. If this check fails after compatibility recreation, onboarding prints failure diagnostics and attempts to restore the pre-patch container before it exits. If that rollback fails, onboarding reports that the pre-patch container was not restored and prints container-cleanup guidance. The local-provider failure output includes the endpoint and recovery steps before the first agent prompt. GPU-proof diagnostics are captured before rollback and can also print cleanup guidance before the final container state is known, so inspect the sandbox and its labeled Docker containers before running a deletion command.

Remote NVIDIA NIM and other compatible endpoints receive their provider validation during onboarding but do not receive this local-provider post-ready check. For those routes, continue to the final route check, then use the status command and a short agent request after onboarding.

Understand Final Route Checks

For NVIDIA Endpoints, onboarding verifies the sandbox-attached provider through the native endpoint. It does not request the shared inference.local models route.

For shared-route providers, onboarding first requests https://inference.local/v1/models from inside the sandbox after policy and process recovery. Each attempt allows 2 seconds for the route to return an HTTP response. A transport failure, an unsupported models-route HTTP 404, or HTTP 5xx leaves the onboarding session retryable at final verification. For a supported OpenRouter agent, onboarding accepts the adapter’s catalog 404 only after a bounded inference request validates the selected model. Onboarding retries any other 404 within its startup budget, then reports the missing model catalog and exits nonzero if the failure persists. Confirm the provider and model configuration and restore the endpoint’s /v1/models route. Run nemohermes onboard --resume to verify the retained session again without rebuilding a healthy sandbox.

Provider setup still performs its own model, credential, and endpoint validation before this final route check. Use the status command and a short agent request after onboarding to verify ongoing availability and model responses.

Send a Short Agent Request

Connect to the sandbox and send a short request before starting long-running work. A successful response proves that the configured model can serve an agent request through the selected provider path.

If the status route is reachable but a tool action returns JSON as normal assistant text, troubleshoot structured tool calling instead of the network route.