InferenceManage Inference

Switch Inference Models

View as Markdown

Change the model on an existing provider route with a runtime update or a fresh sandbox recreation. The command requires both the provider and model even when the provider does not change.

Check the Current Route

Read the live route before changing it.

$nemoclaw inference get

Use the provider ID in the output with the new model ID.

Change the Model at Runtime

Runtime model changes update the OpenShell route and synchronize the selected running agent configuration. Pass --sandbox <name> when you do not want to use the default sandbox.

Open the sandbox for managed configuration changes, switch the model, and restore shields.

$nemoclaw <name> shields down
$nemoclaw inference set --provider <provider> --model <new-model> --sandbox <name>
$nemoclaw <name> shields up

For a compatible endpoint, omit --endpoint-url when the durable registry entry already contains the endpoint and API-family metadata. NemoClaw reuses the recorded route and does not repoint the gateway. You can also re-supply the same endpoint URL when the registry records that onboarding established it. NemoClaw requires an exact canonical match and does not extend that trust to a different URL or an endpoint recorded by inference set. If the route metadata is incomplete, NemoClaw stops and tells you to re-run onboarding.

Account for Shared Gateways

One gateway has one live route even when NemoClaw records different intended routes for its sandboxes. Review Use Shared Gateway Routes before changing a model that another registered sandbox shares. Runtime inference set remains fail-closed when the requested change conflicts with another registered sandbox or when route metadata is incomplete.

Verify the Change

Read the route again after the switch.

$nemoclaw inference get

Use a sandbox-route verification when you also need to prove that the agent can reach the model.