nemo_gym.client.openai_client

The OpenAI-compatible client that Gym servers use to call model endpoints, with retries.

The Responses API and Chat Completions types it sends and receives live in nemo_gym.core.responses_api_types.

Module Contents

Classes

Name	Description
`NeMoGymAsyncOpenAI`	This is just a stub class that wraps around aiohttp
`PermanentEndpointError`	The upstream key is spent or unauthorized; further calls on this client skip the wire.

Functions

Name	Description
`_body_read_cause`	The failure an error reply’s failed body read is linked to: its observed attempt or the forwarded failure.
`_decode_error_text`	-
`_error_body_is_permanent_auth`	True when a 401/403 body is a revoked or invalid key, not a one-off denial.
`_error_body_is_permanent_quota`	True when a 429 body is a spent key, not a transient rate limit.
`_model_attempt_failure`	The model boundary’s failure for one HTTP attempt of a model call (`operation_id`).
`_parsed_error_codes`	error.code / error.type tokens from an OpenAI-style error body.
`_raise_exhausted`	`raise_for_status` for the last attempt; its error’s explicit cause is that attempt’s observed failure.
`_read_error_body`	Read an error reply’s body once; a failed read names the attempt that returned it as its explicit cause.
`_retry_after_seconds`	Seconds a Retry-After header asks for, as a number or an HTTP date; None without a usable one.
`_retry_wait_seconds`	Full-jitter exponential backoff, or the provider’s Retry-After, both capped.
`_rollout_request_id`	A request id that names the rollout, for calls made inside a rollout context.
`_timed_out_waiting_for_response`	Whether the traceback places a timeout in aiohttp response processing.

Data

MODEL_RESPONSE_TIMEOUT_REASON

PERMANENT_AUTH_CODES

PERMANENT_QUOTA_CODES

RATE_LIMIT_ERROR_CODES

REQUEST_ID_HEADER

RETRY_BACKOFF_BASE_SECONDS

RETRY_BACKOFF_MAX_EXPONENT

RETRY_BACKOFF_MAX_SECONDS

RETRY_ERROR_CODES

WAIT_OUT_ERROR_CODES

__getattr__

API

class nemo_gym.client.openai_client.NeMoGymAsyncOpenAI()

Bases: BaseModel

This is just a stub class that wraps around aiohttp

_permanent_trip

Optional[tuple[int, bytes, str]] = PrivateAttr(default=None)

api_key

str

base_url

str

connect_timeout_seconds

Optional[float]

default_headers

Dict[str, str]

internal

bool

max_connection_retries

Optional[int]

max_http_attempts

int = Field(default=MAX_NUM_TRIES, ge=1)

max_num_tries

Optional[Literal[1]]

max_rate_limit_wait_seconds

float

request_timeout_seconds

Optional[float]

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._raise_for_status(
    response: nemo_gym.client.http_client.ClientResponse,
    request_kwargs: typing.Dict[str, typing.Any]
) -> None

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._raise_permanent_error(
    cause: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent] = None
) -> None

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._read_response_json(
    response: nemo_gym.client.http_client.ClientResponse
) -> typing.Dict[str, typing.Any]

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._request(
    request_kwargs: typing.Dict = {}
) -> nemo_gym.client.http_client.ClientResponse

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._request_with_retry(
    request_kwargs: typing.Dict = {}
) -> nemo_gym.client.http_client.ClientResponse

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI._trip_permanent_error(
    status: int,
    content: bytes | str,
    url: typing.Any,
    cause: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent] = None
) -> None

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_chat_completion(
    _resend_after_disconnect: bool = True,
    kwargs = {}
) -> typing.Dict[str, typing.Any]

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_completion(
    kwargs = {}
)

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_embeddings(
    kwargs = {}
)

async

POST {base_url}/embeddings with kwargs as the JSON body and return the parsed response dict.

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_models()

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_response(
    kwargs = {}
)

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.create_tokenize(
    kwargs = {}
)

async

nemo_gym.client.openai_client.NeMoGymAsyncOpenAI.validate_timeout_pair() -> nemo_gym.client.openai_client.NeMoGymAsyncOpenAI

class nemo_gym.client.openai_client.PermanentEndpointError()

Bases: ClientResponseError

The upstream key is spent or unauthorized; further calls on this client skip the wire.

nemo_gym.client.openai_client._body_read_cause(
    response: nemo_gym.client.http_client.ClientResponse,
    failure: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent]
) -> typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent]

The failure an error reply’s failed body read is linked to: its observed attempt or the forwarded failure.

A forwarded Gym failure keeps its original ids; no local attempt is added for it.

nemo_gym.client.openai_client._decode_error_text(
    content: bytes | str
) -> str

nemo_gym.client.openai_client._error_body_is_permanent_auth(
    content: bytes | str
) -> bool

True when a 401/403 body is a revoked or invalid key, not a one-off denial.

nemo_gym.client.openai_client._error_body_is_permanent_quota(
    content: bytes | str
) -> bool

True when a 429 body is a spent key, not a transient rate limit.

nemo_gym.client.openai_client._model_attempt_failure(
    reason: str,
    operation_id: str,
    attempt_id: str,
    http_status: typing.Optional[int] = None,
    foreign_error: typing.Optional[nemo_gym.core.failure_records.SdkErrorDetails] = None,
    request_upload_complete: bool | None = None
) -> nemo_gym.core.typed_exceptions.NgFailureEvent

The model boundary’s failure for one HTTP attempt of a model call (operation_id).

nemo_gym.client.openai_client._parsed_error_codes(
    content: bytes | str
) -> list[str]

error.code / error.type tokens from an OpenAI-style error body.

nemo_gym.client.openai_client._raise_exhausted(
    response: nemo_gym.client.http_client.ClientResponse,
    content: typing.Optional[bytes],
    last_failure: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent]
) -> None

async

raise_for_status for the last attempt; its error’s explicit cause is that attempt’s observed failure.

A cause raise_for_status set itself (a Gym server’s rebuilt failure) is kept.

nemo_gym.client.openai_client._read_error_body(
    response: nemo_gym.client.http_client.ClientResponse,
    failure: typing.Optional[nemo_gym.core.typed_exceptions.NgFailureEvent]
) -> bytes

async

Read an error reply’s body once; a failed read names the attempt that returned it as its explicit cause.

nemo_gym.client.openai_client._retry_after_seconds(
    response: nemo_gym.client.http_client.ClientResponse
) -> typing.Optional[float]

Seconds a Retry-After header asks for, as a number or an HTTP date; None without a usable one.

nemo_gym.client.openai_client._retry_wait_seconds(
    tries: int,
    response: nemo_gym.client.http_client.ClientResponse
) -> float

Full-jitter exponential backoff, or the provider’s Retry-After, both capped.

nemo_gym.client.openai_client._rollout_request_id(
    headers: collections.abc.Mapping[str, str]
) -> typing.Optional[str]

A request id that names the rollout, for calls made inside a rollout context.

vLLM takes its request id from X-Request-Id and records it as gen_ai.request.id on the request’s span and in its request log, so an engine span or log line can be matched to a rollout without walking the trace. The suffix keeps every call distinct, including across server workers that cannot share a counter. A caller’s own header wins.

nemo_gym.client.openai_client._timed_out_waiting_for_response(
    error: BaseException
) -> bool

Whether the traceback places a timeout in aiohttp response processing.

This alone does not establish a completed upload; the caller must require that evidence separately.

With only a total timeout, aiohttp raises the same TimeoutError while connecting or waiting for a pooled connection, so where it fired decides. Any other shape is not a response timeout.

nemo_gym.client.openai_client.MODEL_RESPONSE_TIMEOUT_REASON = 'Model request timed out waiting for the response'

The failure_reason of a model attempt that was sent and whose response did not arrive in time.

nemo_gym.client.openai_client.PERMANENT_AUTH_CODES = ('invalid_api_key', 'invalid_api_token', 'authentication_error')

nemo_gym.client.openai_client.PERMANENT_QUOTA_CODES = ('budget_exceeded', 'insufficient_quota')

nemo_gym.client.openai_client.RATE_LIMIT_ERROR_CODES = [429, 502, 503, 504, 520]

nemo_gym.client.openai_client.REQUEST_ID_HEADER = 'X-Request-Id'

nemo_gym.client.openai_client.RETRY_BACKOFF_BASE_SECONDS = 1.0

nemo_gym.client.openai_client.RETRY_BACKOFF_MAX_EXPONENT = 6

nemo_gym.client.openai_client.RETRY_BACKOFF_MAX_SECONDS = 60.0

nemo_gym.client.openai_client.RETRY_ERROR_CODES = RATE_LIMIT_ERROR_CODES + [404, 408, 500]

nemo_gym.client.openai_client.WAIT_OUT_ERROR_CODES = (429, 503)

nemo_gym.client.openai_client.__getattr__ = moved_names_getattr(__name__)