nemo_gym.h2_ping_sidecar.launcher

Start, supervise and stop the h2-ping-sidecar for gym env start, and point model URLs at it.

H2PingSidecarManager is the entry point. It runs the proxy as an ordinary child process of the process that starts the Gym servers (RunHelper), so the proxy is always on the same node as the model servers that call it, including when RunHelper itself runs inside a Ray actor.

Module Contents

Classes

Name	Description
`H2PingSidecarManager`	Owns the sidecar processes a run started.
`NodeSidecars`	The sidecar processes this run owns: one child process per instance.
`RewrittenUrl`	-
`SidecarError`	The sidecar could not be configured, started or kept running.

Functions

Name	Description
`_local_go_version`	`go env GOVERSION` of the Go on PATH, or an empty string when there is none.
`_local_url`	-
`_log_tail`	-
`_source_digest`	Short hash of the Go sources the binary is built from.
`build_sidecar`	Compile the bundled Go source into `output_path`.
`default_binary_path`	Where the sidecar built from the current sources lives: `<cache>/<sources hash>/<go version>/`.
`h2_ping_sidecar_config_from_global_config`	Build the config from the optional top-level `sidecar:` block; absent means disabled.
`resolve_binary`	The sidecar executable, building it first when allowed and missing.
`resolve_log_dir`	-
`rewrite_base_urls`	Point every model URL aimed at an instance’s upstream at that instance’s local address, in place.
`sidecar_command`	-
`sidecar_env`	Environment for the child: ours, plus the Go runtime limits from the config.
`start_h2_ping_sidecar`	Start the sidecar when `sidecar.enabled` and point model URLs at it.

Data

SIDECAR_BINARY_NAME

SIDECAR_SOURCE_DIR

_BASE_URL_KEY_SUFFIX

_BUILD_TIMEOUT_SEC

_KILL_REAP_TIMEOUT_SEC

_LOG_TAIL_LINES

_MODEL_SERVER_TYPE

_READY_POLL_INTERVAL_SEC

_STOP_SLACK_SEC

API

class nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager(
    config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig,
    instances: typing.List[nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig],
    binary: pathlib.Path,
    log_dir: str
)

Owns the sidecar processes a run started.

_binary

= str(binary)

_sidecars

Optional[NodeSidecars] = None

log_dir

str

nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager.check() -> None

Raise if any sidecar has stopped. Called from Gym’s poll loop.

nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager.start() -> None

nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager.stop() -> None

class nemo_gym.h2_ping_sidecar.launcher.NodeSidecars(
    config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig,
    instances: typing.List[nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig],
    binary: str,
    log_dir: str
)

The sidecar processes this run owns: one child process per instance.

_host

= socket.gethostname()

_log_paths

Dict[str, str] = {}

_processes

Dict[str, Popen] = {}

_ready_id

= uuid4().hex

nemo_gym.h2_ping_sidecar.launcher.NodeSidecars._ready_path(
    instance: nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig
) -> str

nemo_gym.h2_ping_sidecar.launcher.NodeSidecars._spawn(
    instance: nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig
) -> None

nemo_gym.h2_ping_sidecar.launcher.NodeSidecars._wait_ready(
    instance: nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig
) -> None

nemo_gym.h2_ping_sidecar.launcher.NodeSidecars.failures() -> typing.List[str]

One message per sidecar that is no longer running.

nemo_gym.h2_ping_sidecar.launcher.NodeSidecars.start() -> typing.List[str]

Launch every instance and wait until each has bound its port; returns one summary line each.

nemo_gym.h2_ping_sidecar.launcher.NodeSidecars.stop() -> None

SIGTERM every sidecar (it drains in-flight requests), then SIGKILL stragglers.

class nemo_gym.h2_ping_sidecar.launcher.RewrittenUrl(
    location: str,
    old: str,
    new: str
)

Dataclass

location

str

new

str

old

str

class nemo_gym.h2_ping_sidecar.launcher.SidecarError()

Bases: RuntimeError

The sidecar could not be configured, started or kept running.

nemo_gym.h2_ping_sidecar.launcher._local_go_version() -> str

go env GOVERSION of the Go on PATH, or an empty string when there is none.

nemo_gym.h2_ping_sidecar.launcher._local_url(
    url: str,
    instances_by_origin: typing.Dict[str, nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig],
    location: str
) -> typing.Optional[str]

nemo_gym.h2_ping_sidecar.launcher._log_tail(
    log_path: str
) -> str

nemo_gym.h2_ping_sidecar.launcher._source_digest() -> str

Short hash of the Go sources the binary is built from.

A binary built from older sources must never be reused, so the hash is part of the cache path.

nemo_gym.h2_ping_sidecar.launcher.build_sidecar(
    output_path: pathlib.Path
) -> None

Compile the bundled Go source into output_path.

The compiler writes to a private temporary file that is moved into place afterwards, so a concurrent run sharing the cache never executes a half-written binary.

nemo_gym.h2_ping_sidecar.launcher.default_binary_path(
    global_config_dict: typing.Any
) -> pathlib.Path

Where the sidecar built from the current sources lives: <cache>/<sources hash>/<go version>/.

The proxy is mostly Go’s own net/http and crypto/tls, whose fixes ship as Go releases, so a binary built by a Go that has since been replaced is not reused: with Go on PATH the exact version is part of the path. Without Go nothing can be built, so any binary already built from these sources is used (the newest one), which lets a node without Go run a prebuilt cache.

nemo_gym.h2_ping_sidecar.launcher.h2_ping_sidecar_config_from_global_config(
    global_config_dict: typing.Any
) -> nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig

Build the config from the optional top-level sidecar: block; absent means disabled.

nemo_gym.h2_ping_sidecar.launcher.resolve_binary(
    config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig,
    global_config_dict: typing.Any
) -> pathlib.Path

The sidecar executable, building it first when allowed and missing.

nemo_gym.h2_ping_sidecar.launcher.resolve_log_dir(
    config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig,
    global_config_dict: typing.Any
) -> str

nemo_gym.h2_ping_sidecar.launcher.rewrite_base_urls(
    global_config_dict: omegaconf.DictConfig,
    instances: typing.List[nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig]
) -> typing.List[nemo_gym.h2_ping_sidecar.launcher.RewrittenUrl]

Point every model URL aimed at an instance’s upstream at that instance’s local address, in place.

Covers any *base_url key (a string or a list of strings) of a model server under responses_api_models; top-level keys such as policy_base_url are left as they are. The resolved value is what gets matched, so a URL that comes from a resolver such as ${oc.env:JUDGE_URL} or from an alias such as ${policy_base_url} is routed through the sidecar too, and is materialized as a literal in memory for that field only. A field that aliases a whole list is rewritten as a resolved copy, so the list it points at is never changed. Raises SidecarError for a value that is not a parseable URL.

nemo_gym.h2_ping_sidecar.launcher.sidecar_command(
    binary: str,
    config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig,
    instance: nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig,
    ready_path: str
) -> typing.List[str]

nemo_gym.h2_ping_sidecar.launcher.sidecar_env(
    config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig
) -> typing.Dict[str, str]

Environment for the child: ours, plus the Go runtime limits from the config.

nemo_gym.h2_ping_sidecar.launcher.start_h2_ping_sidecar(
    global_config_dict: omegaconf.DictConfig
) -> typing.Optional[nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager]

Start the sidecar when sidecar.enabled and point model URLs at it.

Returns the manager to poll and stop later, or None when the feature is off. Raises SidecarError (after stopping anything it started) when it cannot be brought up. Call it after initialize_ray() and before the config is serialized for the servers.

Everything that can be checked without side effects is checked first, so a bad config fails with no process started and the live config untouched. Once the proxy is running, any later failure or interrupt stops it before the error propagates, because nothing else owns it yet.

nemo_gym.h2_ping_sidecar.launcher.SIDECAR_BINARY_NAME = 'h2-ping-sidecar'

nemo_gym.h2_ping_sidecar.launcher.SIDECAR_SOURCE_DIR = Path(__file__).resolve().parent.parent / 'tools' / 'sidecar'

nemo_gym.h2_ping_sidecar.launcher._BASE_URL_KEY_SUFFIX = 'base_url'

nemo_gym.h2_ping_sidecar.launcher._BUILD_TIMEOUT_SEC = 300.0

nemo_gym.h2_ping_sidecar.launcher._KILL_REAP_TIMEOUT_SEC = 5.0

nemo_gym.h2_ping_sidecar.launcher._LOG_TAIL_LINES = 20

nemo_gym.h2_ping_sidecar.launcher._MODEL_SERVER_TYPE = 'responses_api_models'

nemo_gym.h2_ping_sidecar.launcher._READY_POLL_INTERVAL_SEC = 0.1

nemo_gym.h2_ping_sidecar.launcher._STOP_SLACK_SEC = 5.0