InferenceManage Inference

Switch Inference Providers

View as Markdown

Move a sandbox to another provider family while keeping the OpenShell route, agent configuration, and host registry aligned. Use onboarding first when the target provider is not registered.

Find a Registered Provider

List the configured provider credentials when you need the exact provider ID.

$nemohermes credentials list

inference set also accepts installer-facing provider names such as anthropicCompatible, build, and custom. NemoClaw normalizes an accepted alias to its canonical OpenShell provider ID and records the canonical name in the sandbox registry.

If OpenShell cannot find the requested provider, NemoClaw leaves the route and sandbox state unchanged. Run nemohermes onboard to register the provider, then retry the switch.

Switch the Provider at Runtime

Pass the target provider and model together. Pass --sandbox <name> when you do not want to use the default sandbox.

$nemohermes <name> shields down
$nemohermes inference set --provider <provider> --model <model> --sandbox <name>
$nemohermes <name> shields up

For OpenClaw, NemoClaw updates the provider namespace and selected model in the running configuration. Changes within the current API family hot-reload without replacing the gateway process. When the API family changes, NemoClaw commits the route and configuration, then restarts only the OpenClaw gateway and verifies its health.

For Hermes, NemoClaw updates /sandbox/.hermes/config.yaml, including the model, base URL, API-family mode, and OpenShell proxy API-key placeholder. Hermes does not rebuild or restart for this runtime route change.

If the in-sandbox configuration sync fails after the gateway route changes, NemoClaw keeps the gateway and host registry aligned and prints a rebuild hint. Run the rebuild before relying on the running agent.

Use --no-verify only when OpenShell cannot verify the target provider at switch time and you have already confirmed its provider and credential. This flag does not bypass shared-gateway compatibility checks.

Recreate a Deep Agents Sandbox

Handle Compatible Endpoints

When moving from another provider family to compatible-endpoint or compatible-anthropic-endpoint, provide the trusted endpoint URL and enough API metadata to record the complete route identity. Run this command within the shields-down window shown in the runtime switching workflow.

$nemohermes inference set \
>  --provider compatible-endpoint \
>  --model <model-name> \
>  --endpoint-url <trusted-url> \
>  --inference-api openai-completions \
>  --sandbox <name>

Supported API-family values are openai-completions, anthropic-messages, and openai-responses. For a Hermes compatible-anthropic-endpoint target, omit --inference-api because NemoClaw selects openai-completions. An explicit different API family is rejected for that route.

To point a sandbox at a different custom endpoint, re-run onboarding with the new endpoint. A rebuild reuses the recorded endpoint and cannot change it.

Account for Shared Gateways

Every sandbox registered on one OpenShell gateway must use the same provider and model. Custom routes must also use the same normalized endpoint and API family. NemoClaw includes stopped sandboxes because they depend on that route when restarted.

When a route conflicts, NemoClaw exits before changing the gateway, agent configuration, or host registry and names the affected sandboxes. Align the routes, remove a conflicting sandbox, or onboard it with another NEMOCLAW_GATEWAY_PORT.

Do not run openshell inference set directly on a shared NemoClaw gateway because that bypasses registry compatibility checks.