> This page is for NeMo Relay, version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v0.11.2) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.11.2: https://docs.nvidia.com/nemo/relay/v0.11.2/llms.txt
> - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.

# Provider Connections

> Choose provider endpoints, transport settings, response timeouts, and route destinations.

Configure where Relay forwards OpenAI-compatible and Anthropic-compatible
requests. This page covers base URLs, transport requirements, response
timeouts, and the setting used by each gateway route.

If the endpoint requires credentials, continue with
[Provider Authentication](/nemo/relay/dev/nemo-relay-cli/authentication). Configure a
replacement endpoint and its credentials in the same configuration layer:
changing the endpoint clears inherited credentials associated with the
previous URL. Refer to [Configuration](/nemo/relay/dev/nemo-relay-cli/configuration#shared-configuration)
for file selection and precedence.

## Provider Upstreams

Set provider base URLs under `[upstream]` when you want Relay to forward
compatible requests to a proxy, enterprise endpoint, or another provider host:

```toml
[upstream]
openai_base_url = "https://api.openai.com/v1"
anthropic_base_url = "https://api.anthropic.com"
chatgpt_codex_base_url = "https://chatgpt.com/backend-api/codex"
```

Provider base URLs require HTTPS by default. HTTP is allowed for loopback hosts
such as `localhost`, `127.0.0.1`, and `::1`. Relay validates TOML values,
environment variables, and command-line overrides at startup. Base URLs must not
include user information, query strings, or fragments.

## Credential-Free HTTP Model Upstreams

For a personal gateway or managed worker connected to an unauthenticated HTTP inference gateway,
explicitly list the trusted hosts in `config.toml`:

```toml
[upstream]
openai_base_url = "http://inference-gateway.internal:8080/custom/v1"
insecure_http_model_hosts = ["inference-gateway.internal"]
```

> **Warning**
>
> HTTP exposes prompts, tool results, and model responses without encryption or
> transport integrity. Prefer HTTPS. This exception supports unauthenticated model
> upstreams only: Relay removes credential headers and does not inject environment
> API keys or invoke credential helpers for these requests. Model content and
> request query strings are forwarded; do not place credentials in either.

Entries are bare DNS hostnames or IP addresses. Matching is exact, ignores DNS
case and one trailing DNS dot, and does not trust subdomains or DNS-resolved
addresses automatically. Schemes, ports, paths, user information, wildcards, and
alternate numeric IPv4 spellings are invalid entries. A listed host may use custom
ports and paths, but each request must still target the origin of a configured
OpenAI or Anthropic upstream. A higher-priority configuration list replaces the
lower list; an empty list revokes trust.

For hardened deployments, set an explicit empty list in the system `config.toml`
to prevent lower-priority files from enabling remote HTTP model hosts:

```toml
[upstream]
insecure_http_model_hosts = []
```

System configuration has the highest file-level priority. Omitting this setting
preserves the lower-priority list; specifying `[]` replaces it. The interactive
editor uses the same schema-backed list controls as plugin configuration, and
Reset or Clear saves an explicit empty list.

Remote HTTP upstreams conflict with that provider's configured `api_key`,
`auth_header`, or `api_key_helper`, including `NEMO_RELAY_*_AUTH_HEADER` environment
overrides. Startup fails on a conflict. Raw `OPENAI_API_KEY` and
`ANTHROPIC_API_KEY` environment fallbacks are ignored on these requests. Changing
an upstream URL continues to clear credentials associated with the previous URL.

Only content and protocol headers are forwarded: `Accept`, `Accept-Encoding`,
`Content-Type`, `Content-Length`, `Content-Encoding`, `Content-Language`,
`Anthropic-Version`, `Anthropic-Beta`, and `OpenAI-Beta`. Cookies, authorization,
API keys, and custom headers are removed, including headers added by middleware.
The HTTP client does not use environment proxies, follow redirects, or retry
automatically. Existing HTTPS and loopback behavior is unchanged.

This opt-in supports HTTP requests and SSE streaming in personal gateways,
managed workers, and daemon pass-through routing. Managed workers load user
configuration followed by system configuration, so administrators can deny remote
HTTP model hosts with an explicit empty system list. Credential fields in request
trailers are filtered by the same header allowlist.
Remote HTTP WebSockets, ChatGPT credential backends,
caller-selected destinations, MCP endpoints, and observability exporters retain
their existing transport policies. The host list has no environment-variable or
CLI override; configure it explicitly in the file. You can also edit it through
`nemo-relay config edit` under **Provider upstreams**.

## Overrides, Timeouts, and Routing

For the ChatGPT Codex upstream, `--chatgpt-codex-base-url` overrides
`NEMO_RELAY_CHATGPT_CODEX_BASE_URL`, which overrides the TOML value. The flag
is available at the top level and on `run`; an explicit `run` flag overrides
an inherited top-level flag.

Provider waits have no deadline by default so long compaction can finish. Caller
cancellation ends the request. Set `[upstream] response_timeout_secs` to a
positive number of seconds to limit response-header waits. In the standalone
gateway, this also limits each idle response-body read; successful reads restart
the timer. Set the value to `0` to disable these limits. Connection establishment
and short plugin callbacks retain separate deadlines.

Relay normally chooses the upstream from the request path, regardless of which
compatible client sends the request. For standard provider paths, gateway routes
map to settings as follows:

| Gateway Endpoint                                                          | Upstream Setting                                                                                                                                                                                                       |
| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `/responses`, `/v1/responses`                                             | `openai_base_url`                                                                                                                                                                                                      |
| `/chat/completions`, `/v1/chat/completions`                               | `openai_base_url`                                                                                                                                                                                                      |
| `/responses/compact`, `/v1/responses/compact`                             | `openai_base_url`                                                                                                                                                                                                      |
| `/realtime/calls`, `/v1/realtime/calls`                                   | `openai_base_url`; with the default base, recognized ChatGPT credentials use the ChatGPT backend                                                                                                                       |
| `/live`, `/v1/live`, `/live/sessions`, `/v1/live/sessions`                | `openai_base_url`                                                                                                                                                                                                      |
| `/realtime`, `/v1/realtime`                                               | `openai_base_url`                                                                                                                                                                                                      |
| `/live/CALL_ID`, `/v1/live/CALL_ID`                                       | `openai_base_url`                                                                                                                                                                                                      |
| `/live/sessions/SESSION_ID/attach`, `/v1/live/sessions/SESSION_ID/attach` | `openai_base_url`                                                                                                                                                                                                      |
| `/images/edits`, `/v1/images/edits`                                       | `openai_base_url`                                                                                                                                                                                                      |
| `/memories/trace_summarize`, `/v1/memories/trace_summarize`               | `openai_base_url`                                                                                                                                                                                                      |
| `/alpha/search`, `/v1/alpha/search`                                       | `openai_base_url`                                                                                                                                                                                                      |
| `/models`, `/v1/models`                                                   | `anthropic_base_url` with `anthropic-version` or `anthropic-beta`; otherwise `openai_base_url`. With the default OpenAI base, recognized ChatGPT credentials and no replacement credential select the ChatGPT backend. |
| `/v1/messages`, `/v1/messages/count_tokens`                               | `anthropic_base_url`                                                                                                                                                                                                   |

For credential-dependent routing of `/backend-api/codex/responses`,
`/backend-api/codex/responses/compact`, and
`/backend-api/codex/realtime/calls`, see
[Codex ChatGPT Credentials](/nemo/relay/dev/nemo-relay-cli/authentication#codex-chatgpt-credentials).

After choosing the endpoint, configure its
[credentials](/nemo/relay/dev/nemo-relay-cli/authentication) and inspect the effective
configuration with [doctor](/nemo/relay/dev/nemo-relay-cli/troubleshooting#check-configuration-and-exporters).