InferenceManage Inference

Switch Inference Models

View as Markdown

Change the model on an existing provider route with a runtime update or a fresh sandbox recreation. The command requires both the provider and model even when the provider does not change.

Check the Current Route

Read the live route before changing it.

$nemohermes inference get

Use the provider ID in the output with the new model ID.

Change the Model at Runtime

Runtime model changes update the OpenShell route and synchronize the selected running agent configuration. Pass --sandbox <name> when you do not want to use the default sandbox.

Open the sandbox for managed configuration changes, switch the model, and restore shields.

$nemohermes <name> shields down
$nemohermes inference set --provider <provider> --model <new-model> --sandbox <name>
$nemohermes <name> shields up

For a compatible endpoint, omit --endpoint-url when the durable registry entry already contains the endpoint and API-family metadata. NemoClaw reuses the recorded route and does not repoint the gateway. If the route metadata is incomplete, NemoClaw stops and tells you to re-run onboarding.

Account for Shared Gateways

OpenShell exposes one inference route per gateway. Every registered sandbox on the same gateway, including stopped sandboxes, must record the same provider and model. NemoClaw exits before changing state when the requested model conflicts with another sandbox on the gateway.

Align the sandboxes to one route, remove a sandbox you no longer need, or onboard a sandbox with another NEMOCLAW_GATEWAY_PORT.

Verify the Change

Read the route again after the switch.

$nemohermes inference get

Use a sandbox-route verification when you also need to prove that the agent can reach the model.