Provider Connections
Configure where Relay forwards OpenAI-compatible and Anthropic-compatible requests. This page covers base URLs, transport requirements, response timeouts, and the setting used by each gateway route.
If the endpoint requires credentials, continue with Provider Authentication. Configure a replacement endpoint and its credentials in the same configuration layer: changing the endpoint clears inherited credentials associated with the previous URL. Refer to Configuration for file selection and precedence.
Provider Upstreams
Set provider base URLs under [upstream] when you want Relay to forward
compatible requests to a proxy, enterprise endpoint, or another provider host:
Provider base URLs require HTTPS by default. HTTP is allowed for loopback hosts
such as localhost, 127.0.0.1, and ::1. Relay validates TOML values,
environment variables, and command-line overrides at startup. Base URLs must not
include user information, query strings, or fragments.
Credential-Free HTTP Model Upstreams
For a personal gateway or managed worker connected to an unauthenticated HTTP inference gateway,
explicitly list the trusted hosts in config.toml:
HTTP exposes prompts, tool results, and model responses without encryption or transport integrity. Prefer HTTPS. This exception supports unauthenticated model upstreams only: Relay removes credential headers and does not inject environment API keys or invoke credential helpers for these requests. Model content and request query strings are forwarded; do not place credentials in either.
Entries are bare DNS hostnames or IP addresses. Matching is exact, ignores DNS case and one trailing DNS dot, and does not trust subdomains or DNS-resolved addresses automatically. Schemes, ports, paths, user information, wildcards, and alternate numeric IPv4 spellings are invalid entries. A listed host may use custom ports and paths, but each request must still target the origin of a configured OpenAI or Anthropic upstream. A higher-priority configuration list replaces the lower list; an empty list revokes trust.
For hardened deployments, set an explicit empty list in the system config.toml
to prevent lower-priority files from enabling remote HTTP model hosts:
System configuration has the highest file-level priority. Omitting this setting
preserves the lower-priority list; specifying [] replaces it. The interactive
editor uses the same schema-backed list controls as plugin configuration, and
Reset or Clear saves an explicit empty list.
Remote HTTP upstreams conflict with that provider’s configured api_key,
auth_header, or api_key_helper, including NEMO_RELAY_*_AUTH_HEADER environment
overrides. Startup fails on a conflict. Raw OPENAI_API_KEY and
ANTHROPIC_API_KEY environment fallbacks are ignored on these requests. Changing
an upstream URL continues to clear credentials associated with the previous URL.
Only content and protocol headers are forwarded: Accept, Accept-Encoding,
Content-Type, Content-Length, Content-Encoding, Content-Language,
Anthropic-Version, Anthropic-Beta, and OpenAI-Beta. Cookies, authorization,
API keys, and custom headers are removed, including headers added by middleware.
The HTTP client does not use environment proxies, follow redirects, or retry
automatically. Existing HTTPS and loopback behavior is unchanged.
This opt-in supports HTTP requests and SSE streaming in personal gateways,
managed workers, and daemon pass-through routing. Managed workers load user
configuration followed by system configuration, so administrators can deny remote
HTTP model hosts with an explicit empty system list. Credential fields in request
trailers are filtered by the same header allowlist.
Remote HTTP WebSockets, ChatGPT credential backends,
caller-selected destinations, MCP endpoints, and observability exporters retain
their existing transport policies. The host list has no environment-variable or
CLI override; configure it explicitly in the file. You can also edit it through
nemo-relay config edit under Provider upstreams.
Overrides, Timeouts, and Routing
For the ChatGPT Codex upstream, --chatgpt-codex-base-url overrides
NEMO_RELAY_CHATGPT_CODEX_BASE_URL, which overrides the TOML value. The flag
is available at the top level and on run; an explicit run flag overrides
an inherited top-level flag.
Provider waits have no deadline by default so long compaction can finish. Caller
cancellation ends the request. Set [upstream] response_timeout_secs to a
positive number of seconds to limit response-header waits. In the standalone
gateway, this also limits each idle response-body read; successful reads restart
the timer. Set the value to 0 to disable these limits. Connection establishment
and short plugin callbacks retain separate deadlines.
Relay normally chooses the upstream from the request path, regardless of which compatible client sends the request. For standard provider paths, gateway routes map to settings as follows:
For credential-dependent routing of /backend-api/codex/responses,
/backend-api/codex/responses/compact, and
/backend-api/codex/realtime/calls, see
Codex ChatGPT Credentials.
After choosing the endpoint, configure its credentials and inspect the effective configuration with doctor.