InferenceCustom Endpoints

Meet Custom Endpoint Security Requirements

View as Markdown

NemoClaw keeps provider credentials on the host and validates explicit custom endpoint URLs before saving them through security-sensitive configuration paths.

Protect Provider Credentials

The agent inside the sandbox sends requests to inference.local instead of connecting to the upstream endpoint directly. OpenShell forwards the traffic and injects the provider credential at egress. The sandbox does not receive the raw API key.

Use COMPATIBLE_API_KEY for an OpenAI-compatible endpoint that requires authentication. For an HTTP endpoint on localhost, 127.0.0.1, or [::1] and port 8000, 11434, or 11435, you can select no authentication. For non-interactive onboarding of that OpenAI-compatible endpoint, set NEMOCLAW_COMPATIBLE_AUTH_MODE=none. Use COMPATIBLE_ANTHROPIC_API_KEY for a custom Anthropic-compatible endpoint. Anthropic-compatible onboarding requires a non-empty value even when the upstream server does not authenticate requests. Use a non-empty placeholder such as dummy for an unauthenticated Anthropic-compatible endpoint.

Understand URL Validation

Explicit endpoint URLs saved through Hermes Provider setup, inference set, host-side config set, or a direct blueprint run must pass host-side server-side request forgery validation. NemoClaw rejects loopback, link-local, private, internal, reserved, and special-purpose addresses in these paths. Blocked special-purpose targets include documentation, translation, benchmarking, multicast, and metadata destinations. It also rejects public hostnames that resolve to a blocked address.

A direct blueprint run rejects endpoint URLs that contain userinfo, such as an embedded username or password, before DNS resolution. Configure the provider credential separately instead of putting it in the endpoint URL.

Managed provider defaults that do not provide an explicit custom endpoint through these paths are unaffected.

Custom endpoint onboarding has one narrower operator-controlled exception for corporate inference gateways. Set NEMOCLAW_TRUSTED_PRIVATE_HOSTS to a comma-separated list of exact hostnames or IP literals to admit an endpoint on RFC1918, carrier-grade network address translation (CGNAT), or IPv6 unique local address space. NemoClaw still resolves DNS and pins outbound validation to the complete canonical address set. An exact trusted host can return both public and supported private addresses. NemoClaw pins every canonical answer. If any answer is a disallowed private, reserved, or special-purpose address, validation rejects the endpoint instead of discarding that answer. Wildcard or suffix matches and resolver failures also remain blocked. This allowlist does not relax direct blueprint, config set, or unrelated persisted-URL validation.

NEMOCLAW_TRUSTED_PRIVATE_INFERENCE_HOSTS remains an inference-only compatibility alias. Inference onboarding combines exact entries from the generic variable and the compatibility alias. New configurations should use NEMOCLAW_TRUSTED_PRIVATE_HOSTS.

After onboarding records an admitted custom endpoint, inference set accepts that same canonical URL for a model change without resolving it again. The registry must record onboarding as the endpoint source, and the supplied URL must match exactly after normalization. Legacy entries without a source, endpoints recorded by inference set, and different URLs still pass through the full server-side request forgery validation path.

Use a Public Endpoint

For a public HTTP URL, NemoClaw stores the validated IP address so the downstream runtime cannot resolve the hostname again and reach another address.

NemoClaw rejects DNS-backed HTTPS URLs for Hermes Provider setup, host-side config set, and a direct blueprint run because it cannot pin the downstream peer address while preserving TLS Server Name Indication and host validation across the OpenShell runtime boundary. For those paths, use an HTTPS IP-literal endpoint with a certificate valid for that address, or use a public HTTP endpoint if your deployment permits unencrypted traffic.

Use a DNS-Backed HTTPS Endpoint with inference set

nemoclaw inference set --endpoint-url <url> on an already-onboarded sandbox supports a DNS-backed HTTPS custom endpoint through the HTTPS Pin Runtime adapter. After SSRF validation passes, NemoClaw starts a local reverse-proxy adapter on the host that terminates a pinned, SNI-correct outbound TLS connection to the real upstream hostname, re-validating that the resolved peer IP is still public. The sandbox, its OpenShell provider configuration and network policy, and the persisted sandbox registry only ever see the opaque local base http://host.openshell.internal:<port>/route/<id>. The real upstream hostname and path never reach the sandbox or the persisted registry. Host recovery state contains adapter process metadata, the source-subnet policy, opaque route IDs, provider types, non-secret token generation values, and timestamps. It contains no upstream URLs, pinned addresses, or credentials. Endpoint URLs containing userinfo, a query string, or a fragment are rejected rather than stripped or persisted. For an OpenAI-compatible endpoint entered as a bare origin, the adapter preserves the incoming /v1 request path. For an endpoint with a path prefix, the adapter keeps forwarded requests beneath that prefix and rejects traversal-shaped paths. One 30-second total upstream deadline covers connection setup, TLS negotiation, and the complete response; the adapter closes a response that arrives after the deadline instead of relaying it.

Each opaque route has its own sandbox-facing adapter credential, distinct from both the real upstream credential and the host-only control credential; a credential issued for one route cannot authorize another route. NemoClaw does not place upstream route credentials in the adapter child-process environment. After startup, the host CLI registers each route and its credential in adapter memory through an authenticated loopback-only control plane. Before starting or reusing the adapter, NemoClaw inspects the exact IPAM subnets assigned to the openshell-docker network. The adapter accepts route-forwarding and non-control health requests only from loopback or those inspected subnets and returns a not-found response to peers on other private or LAN networks. Authenticated control health and route-registration requests remain loopback-only. NemoClaw refuses to expose the adapter when it cannot discover a valid bridge subnet. Adapter reuse also requires an authenticated health proof for the same source-subnet policy, so a running process with a stale or different policy is replaced. After an adapter restart, routes other than the one that triggered recovery return a recovery-needed response until their original inference set --endpoint-url command is rerun.

Handle a Route Revocation Warning

Before revocation, NemoClaw checks whether another sandbox references the route. If another sandbox references it, NemoClaw keeps the route. If no other sandbox references the route, NemoClaw attempts revocation after it applies the new inference route or completes sandbox deletion. If the reference check or revocation fails, NemoClaw does not restore the previous inference route or the deleted sandbox. NemoClaw prints a warning because the upstream credential can remain in adapter memory.

Revocation must authenticate the running adapter. The control-plane proof binds the source-subnet policy that the adapter used at startup. NemoClaw records that policy in host recovery state when it starts or reuses the adapter, then reads it during revocation. It does not derive the policy again from the current host because a recreated or renumbered bridge would produce a different value.

If an adapter started before NemoClaw recorded the policy, NemoClaw cannot authenticate the adapter for revocation. NemoClaw preserves the route recovery state and reports the failed revocation.

If the sandbox still exists, recover the route as follows:

  1. Rerun the inference set --endpoint-url command that registered the affected route.
  2. Switch away from that route again.

The first command records the source-subnet policy for the running adapter. NemoClaw retries revocation during the second command. When no other sandbox references the route, a completed switch without a revocation warning confirms that the adapter removed it.

Create Snapshots Before Cleanup

NemoClaw attempts to wipe manifest-defined agent state before it deletes a sandbox. OpenShell can retain the per-name persistent volume after sandbox deletion. If the wipe cannot complete, onboarding with the same name can make old files available again. Do not rely on a retained volume as a backup. Before you destroy a remaining sandbox, create a snapshot of any state that you must keep. Refer to Create and Restore Snapshots for instructions.

If the affected sandbox was destroyed or revocation continues to fail, destroy all remaining sandboxes before adapter cleanup. A scoped uninstall that leaves sibling gateways in place preserves the shared adapter and its remaining routes. After you destroy all sandboxes, run nemoclaw uninstall --all-gateway-ports to remove every gateway-port environment and stop the shared adapter. Do not treat its in-memory credentials as purged until the command succeeds, the adapter PID has exited, and no listener remains on NEMOCLAW_HTTPS_PIN_RUNTIME_ADAPTER_PORT (default 11438).

This support is specific to inference set on an already-onboarded sandbox. Hermes Provider setup, host-side config set, and a direct blueprint run still reject DNS-backed HTTPS URLs as described above.

Use the Sandbox Host Alias

NemoClaw accepts http://host.openshell.internal:<port> only when the URL includes an explicit port from 1024 through 65535. This narrow exception supports NemoClaw-managed sandbox-to-host inference routes and host-side config set writes to supported provider baseUrl fields that already use the OpenShell host bridge, including OpenClaw memory-search embedding providers. Generic config set keys continue to reject the bridge host. Accepting this URL shape does not by itself make an arbitrary local service, embedding provider, config key, or network route a supported NemoClaw integration.

For example, a containerized gateway can commonly reach a local server at http://host.openshell.internal:8000/v1. Because this name is a sandbox-internal alias, onboarding skips the host-side endpoint probe. Verify the runtime route after onboarding when you use it.

The exception is intentionally exact: use HTTP, the literal host.openshell.internal name, and an explicit unprivileged port. NemoClaw still rejects HTTPS, privileged ports, subdomains, loopback names, private IP literals, userinfo, query strings, and fragments through this path.