> This page is for version 0.5.1.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.client.openai_client

The OpenAI-compatible client that Gym servers use to call model endpoints, with retries.

The Responses API and Chat Completions types it sends and receives live in `nemo_gym.core.responses_api_types`.

## Module Contents

### Classes

| Name                                                                              | Description                                                                            |
| --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| [`NeMoGymAsyncOpenAI`](#nemo_gym-client-openai_client-NeMoGymAsyncOpenAI)         | This is just a stub class that wraps around aiohttp                                    |
| [`PermanentEndpointError`](#nemo_gym-client-openai_client-PermanentEndpointError) | The upstream key is spent or unauthorized; further calls on this client skip the wire. |

### Functions

| Name                                                                                                | Description                                                                                                |
| --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| [`_body_read_cause`](#nemo_gym-client-openai_client-_body_read_cause)                               | The failure an error reply's failed body read is linked to: its observed attempt or the forwarded failure. |
| [`_decode_error_text`](#nemo_gym-client-openai_client-_decode_error_text)                           | -                                                                                                          |
| [`_error_body_is_permanent_auth`](#nemo_gym-client-openai_client-_error_body_is_permanent_auth)     | True when a 401/403 body is a revoked or invalid key, not a one-off denial.                                |
| [`_error_body_is_permanent_quota`](#nemo_gym-client-openai_client-_error_body_is_permanent_quota)   | True when a 429 body is a spent key, not a transient rate limit.                                           |
| [`_model_attempt_failure`](#nemo_gym-client-openai_client-_model_attempt_failure)                   | The model boundary's failure for one HTTP attempt of a model call (`operation_id`).                        |
| [`_parsed_error_codes`](#nemo_gym-client-openai_client-_parsed_error_codes)                         | error.code / error.type tokens from an OpenAI-style error body.                                            |
| [`_raise_exhausted`](#nemo_gym-client-openai_client-_raise_exhausted)                               | `raise_for_status` for the last attempt; its error's explicit cause is that attempt's observed failure.    |
| [`_read_error_body`](#nemo_gym-client-openai_client-_read_error_body)                               | Read an error reply's body once; a failed read names the attempt that returned it as its explicit cause.   |
| [`_retry_after_seconds`](#nemo_gym-client-openai_client-_retry_after_seconds)                       | Seconds a Retry-After header asks for, as a number or an HTTP date; None without a usable one.             |
| [`_retry_wait_seconds`](#nemo_gym-client-openai_client-_retry_wait_seconds)                         | Full-jitter exponential backoff, or the provider's Retry-After, both capped.                               |
| [`_rollout_request_id`](#nemo_gym-client-openai_client-_rollout_request_id)                         | A request id that names the rollout, for calls made inside a rollout context.                              |
| [`_timed_out_waiting_for_response`](#nemo_gym-client-openai_client-_timed_out_waiting_for_response) | Whether the traceback places a timeout in aiohttp response processing.                                     |

### Data

[`MODEL_RESPONSE_TIMEOUT_REASON`](#nemo_gym-client-openai_client-MODEL_RESPONSE_TIMEOUT_REASON)

[`PERMANENT_AUTH_CODES`](#nemo_gym-client-openai_client-PERMANENT_AUTH_CODES)

[`PERMANENT_QUOTA_CODES`](#nemo_gym-client-openai_client-PERMANENT_QUOTA_CODES)

[`RATE_LIMIT_ERROR_CODES`](#nemo_gym-client-openai_client-RATE_LIMIT_ERROR_CODES)

[`REQUEST_ID_HEADER`](#nemo_gym-client-openai_client-REQUEST_ID_HEADER)

[`RETRY_BACKOFF_BASE_SECONDS`](#nemo_gym-client-openai_client-RETRY_BACKOFF_BASE_SECONDS)

[`RETRY_BACKOFF_MAX_EXPONENT`](#nemo_gym-client-openai_client-RETRY_BACKOFF_MAX_EXPONENT)

[`RETRY_BACKOFF_MAX_SECONDS`](#nemo_gym-client-openai_client-RETRY_BACKOFF_MAX_SECONDS)

[`RETRY_ERROR_CODES`](#nemo_gym-client-openai_client-RETRY_ERROR_CODES)

[`WAIT_OUT_ERROR_CODES`](#nemo_gym-client-openai_client-WAIT_OUT_ERROR_CODES)

[`__getattr__`](#nemo_gym-client-openai_client-__getattr__)

### API

```python
class nemo_gym.client.openai_client.NeMoGymAsyncOpenAI()
```

**Bases:** `BaseModel`

This is just a stub class that wraps around aiohttp

**`_permanent_trip`** `Optional[tuple[int, bytes, str]] = PrivateAttr(default=None)`

---

**`api_key`** `str`

---

**`base_url`** `str`

---

**`connect_timeout_seconds`** `Optional[float]`

---

**`default_headers`** `Dict[str, str]`

---

**`internal`** `bool`

---

**`max_connection_retries`** `Optional[int]`

---

**`max_http_attempts`** `int = Field(default=MAX_NUM_TRIES, ge=1)`

---

**`max_num_tries`** `Optional[Literal[1]]`

---

**`max_rate_limit_wait_seconds`** `float`

---

**`request_timeout_seconds`** `Optional[float]`

---

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._raise_for_status(
    response: nemo_gym.client.http_client.ClientResponse,
    request_kwargs: typing.Dict[str, typing.Any]
) -> None
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._raise_permanent_error(
    cause: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent] = None
) -> None
```

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._read_response_json(
    response: nemo_gym.client.http_client.ClientResponse
) -> typing.Dict[str, typing.Any]
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._request(
    request_kwargs: typing.Dict = {}
) -> nemo_gym.client.http_client.ClientResponse
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._request_with_retry(
    request_kwargs: typing.Dict = {}
) -> nemo_gym.client.http_client.ClientResponse
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._trip_permanent_error(
    status: int,
    content: bytes | str,
    url: typing.Any,
    cause: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent] = None
) -> None
```

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_chat_completion(
    _resend_after_disconnect: bool = True,
    kwargs = {}
) -> typing.Dict[str, typing.Any]
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_completion(
    kwargs = {}
)
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_embeddings(
    kwargs = {}
)
```

async

POST `{base_url}/embeddings` with `kwargs` as the JSON body and return the parsed response dict.

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_models()
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_response(
    kwargs = {}
)
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_tokenize(
    kwargs = {}
)
```

async

```python
nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.validate_timeout_pair() -> nemo_gym.client.openai_client.NeMoGymAsyncOpenAI
```

```python
class nemo_gym.client.openai_client.PermanentEndpointError()
```

**Bases:** `ClientResponseError`

The upstream key is spent or unauthorized; further calls on this client skip the wire.

```python
nemo_gym.client.openai_client._body_read_cause(
    response: nemo_gym.client.http_client.ClientResponse,
    failure: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent]
) -> typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent]
```

The failure an error reply's failed body read is linked to: its observed attempt or the forwarded failure.

A forwarded Gym failure keeps its original ids; no local attempt is added for it.

```python
nemo_gym.client.openai_client._decode_error_text(
    content: bytes | str
) -> str
```

```python
nemo_gym.client.openai_client._error_body_is_permanent_auth(
    content: bytes | str
) -> bool
```

True when a 401/403 body is a revoked or invalid key, not a one-off denial.

```python
nemo_gym.client.openai_client._error_body_is_permanent_quota(
    content: bytes | str
) -> bool
```

True when a 429 body is a spent key, not a transient rate limit.

```python
nemo_gym.client.openai_client._model_attempt_failure(
    reason: str,
    operation_id: str,
    attempt_id: str,
    http_status: typing.Optional[int] = None,
    foreign_error: typing.Optional[nemo_gym.core.failure_records.SdkErrorDetails] = None,
    request_upload_complete: bool | None = None
) -> nemo_gym.core.typed_exceptions.NgFailureEvent
```

The model boundary's failure for one HTTP attempt of a model call (`operation_id`).

```python
nemo_gym.client.openai_client._parsed_error_codes(
    content: bytes | str
) -> list[str]
```

error.code / error.type tokens from an OpenAI-style error body.

```python
nemo_gym.client.openai_client._raise_exhausted(
    response: nemo_gym.client.http_client.ClientResponse,
    content: typing.Optional[bytes],
    last_failure: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent]
) -> None
```

async

`raise_for_status` for the last attempt; its error's explicit cause is that attempt's observed failure.

A cause `raise_for_status` set itself (a Gym server's rebuilt failure) is kept.

```python
nemo_gym.client.openai_client._read_error_body(
    response: nemo_gym.client.http_client.ClientResponse,
    failure: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent]
) -> bytes
```

async

Read an error reply's body once; a failed read names the attempt that returned it as its explicit cause.

```python
nemo_gym.client.openai_client._retry_after_seconds(
    response: nemo_gym.client.http_client.ClientResponse
) -> typing.Optional[float]
```

Seconds a Retry-After header asks for, as a number or an HTTP date; None without a usable one.

```python
nemo_gym.client.openai_client._retry_wait_seconds(
    tries: int,
    response: nemo_gym.client.http_client.ClientResponse
) -> float
```

Full-jitter exponential backoff, or the provider's Retry-After, both capped.

```python
nemo_gym.client.openai_client._rollout_request_id(
    headers: collections.abc.Mapping[str, str]
) -> typing.Optional[str]
```

A request id that names the rollout, for calls made inside a rollout context.

vLLM takes its request id from `X-Request-Id` and records it as `gen_ai.request.id` on
the request's span and in its request log, so an engine span or log line can be matched to a
rollout without walking the trace. The suffix keeps every call distinct, including across
server workers that cannot share a counter. A caller's own header wins.

```python
nemo_gym.client.openai_client._timed_out_waiting_for_response(
    error: BaseException
) -> bool
```

Whether the traceback places a timeout in aiohttp response processing.

This alone does not establish a completed upload; the caller must require that evidence separately.

With only a total timeout, aiohttp raises the same `TimeoutError` while connecting or waiting for a pooled
connection, so where it fired decides. Any other shape is not a response timeout.

```python
nemo_gym.client.openai_client.MODEL_RESPONSE_TIMEOUT_REASON = 'Model request timed out waiting for the response'
```

The `failure_reason` of a model attempt that was sent and whose response did not arrive in time.

```python
nemo_gym.client.openai_client.PERMANENT_AUTH_CODES = ('invalid_api_key', 'invalid_api_token', 'authentication_error')
```

```python
nemo_gym.client.openai_client.PERMANENT_QUOTA_CODES = ('budget_exceeded', 'insufficient_quota')
```

```python
nemo_gym.client.openai_client.RATE_LIMIT_ERROR_CODES = [429, 502, 503, 504, 520]
```

```python
nemo_gym.client.openai_client.REQUEST_ID_HEADER = 'X-Request-Id'
```

```python
nemo_gym.client.openai_client.RETRY_BACKOFF_BASE_SECONDS = 1.0
```

```python
nemo_gym.client.openai_client.RETRY_BACKOFF_MAX_EXPONENT = 6
```

```python
nemo_gym.client.openai_client.RETRY_BACKOFF_MAX_SECONDS = 60.0
```

```python
nemo_gym.client.openai_client.RETRY_ERROR_CODES = RATE_LIMIT_ERROR_CODES + [404, 408, 500]
```

```python
nemo_gym.client.openai_client.WAIT_OUT_ERROR_CODES = (429, 503)
```

```python
nemo_gym.client.openai_client.__getattr__ = moved_names_getattr(__name__)
```