> This page is for version 0.5.1. > For other versions, use one of these documentation indexes: > - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt > - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt > - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt > - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt > - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt > - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt > - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt > - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server. # nemo_gym.h2_ping_sidecar.launcher Start, supervise and stop the h2-ping-sidecar for `gym env start`, and point model URLs at it. `H2PingSidecarManager` is the entry point. It runs the proxy as an ordinary child process of the process that starts the Gym servers (`RunHelper`), so the proxy is always on the same node as the model servers that call it, including when `RunHelper` itself runs inside a Ray actor. ## Module Contents ### Classes | Name | Description | | --------------------------------------------------------------------------------- | -------------------------------------------------------------------- | | [`H2PingSidecarManager`](#nemo_gym-h2_ping_sidecar-launcher-H2PingSidecarManager) | Owns the sidecar processes a run started. | | [`NodeSidecars`](#nemo_gym-h2_ping_sidecar-launcher-NodeSidecars) | The sidecar processes this run owns: one child process per instance. | | [`RewrittenUrl`](#nemo_gym-h2_ping_sidecar-launcher-RewrittenUrl) | - | | [`SidecarError`](#nemo_gym-h2_ping_sidecar-launcher-SidecarError) | The sidecar could not be configured, started or kept running. | ### Functions | Name | Description | | --------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | | [`_local_go_version`](#nemo_gym-h2_ping_sidecar-launcher-_local_go_version) | `go env GOVERSION` of the Go on PATH, or an empty string when there is none. | | [`_local_url`](#nemo_gym-h2_ping_sidecar-launcher-_local_url) | - | | [`_log_tail`](#nemo_gym-h2_ping_sidecar-launcher-_log_tail) | - | | [`_source_digest`](#nemo_gym-h2_ping_sidecar-launcher-_source_digest) | Short hash of the Go sources the binary is built from. | | [`build_sidecar`](#nemo_gym-h2_ping_sidecar-launcher-build_sidecar) | Compile the bundled Go source into `output_path`. | | [`default_binary_path`](#nemo_gym-h2_ping_sidecar-launcher-default_binary_path) | Where the sidecar built from the current sources lives: `<cache>/<sources hash>/<go version>/`. | | [`h2_ping_sidecar_config_from_global_config`](#nemo_gym-h2_ping_sidecar-launcher-h2_ping_sidecar_config_from_global_config) | Build the config from the optional top-level `sidecar:` block; absent means disabled. | | [`resolve_binary`](#nemo_gym-h2_ping_sidecar-launcher-resolve_binary) | The sidecar executable, building it first when allowed and missing. | | [`resolve_log_dir`](#nemo_gym-h2_ping_sidecar-launcher-resolve_log_dir) | - | | [`rewrite_base_urls`](#nemo_gym-h2_ping_sidecar-launcher-rewrite_base_urls) | Point every model URL aimed at an instance's upstream at that instance's local address, in place. | | [`sidecar_command`](#nemo_gym-h2_ping_sidecar-launcher-sidecar_command) | - | | [`sidecar_env`](#nemo_gym-h2_ping_sidecar-launcher-sidecar_env) | Environment for the child: ours, plus the Go runtime limits from the config. | | [`start_h2_ping_sidecar`](#nemo_gym-h2_ping_sidecar-launcher-start_h2_ping_sidecar) | Start the sidecar when `sidecar.enabled` and point model URLs at it. | ### Data [`SIDECAR_BINARY_NAME`](#nemo_gym-h2_ping_sidecar-launcher-SIDECAR_BINARY_NAME) [`SIDECAR_SOURCE_DIR`](#nemo_gym-h2_ping_sidecar-launcher-SIDECAR_SOURCE_DIR) [`_BASE_URL_KEY_SUFFIX`](#nemo_gym-h2_ping_sidecar-launcher-_BASE_URL_KEY_SUFFIX) [`_BUILD_TIMEOUT_SEC`](#nemo_gym-h2_ping_sidecar-launcher-_BUILD_TIMEOUT_SEC) [`_KILL_REAP_TIMEOUT_SEC`](#nemo_gym-h2_ping_sidecar-launcher-_KILL_REAP_TIMEOUT_SEC) [`_LOG_TAIL_LINES`](#nemo_gym-h2_ping_sidecar-launcher-_LOG_TAIL_LINES) [`_MODEL_SERVER_TYPE`](#nemo_gym-h2_ping_sidecar-launcher-_MODEL_SERVER_TYPE) [`_READY_POLL_INTERVAL_SEC`](#nemo_gym-h2_ping_sidecar-launcher-_READY_POLL_INTERVAL_SEC) [`_STOP_SLACK_SEC`](#nemo_gym-h2_ping_sidecar-launcher-_STOP_SLACK_SEC) ### API ```python class nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager( config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig, instances: typing.List[nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig], binary: pathlib.Path, log_dir: str ) ``` Owns the sidecar processes a run started. **`_binary`** `= str(binary)` --- **`_sidecars`** `Optional[NodeSidecars] = None` --- **`log_dir`** `str` --- ```python nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager.check() -> None ``` Raise if any sidecar has stopped. Called from Gym's poll loop. ```python nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager.start() -> None ``` ```python nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager.stop() -> None ``` ```python class nemo_gym.h2_ping_sidecar.launcher.NodeSidecars( config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig, instances: typing.List[nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig], binary: str, log_dir: str ) ``` The sidecar processes this run owns: one child process per instance. **`_host`** `= socket.gethostname()` --- **`_log_paths`** `Dict[str, str] = {}` --- **`_processes`** `Dict[str, Popen] = {}` --- **`_ready_id`** `= uuid4().hex` --- ```python nemo_gym.h2_ping_sidecar.launcher.NodeSidecars._ready_path( instance: nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig ) -> str ``` ```python nemo_gym.h2_ping_sidecar.launcher.NodeSidecars._spawn( instance: nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig ) -> None ``` ```python nemo_gym.h2_ping_sidecar.launcher.NodeSidecars._wait_ready( instance: nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig ) -> None ``` ```python nemo_gym.h2_ping_sidecar.launcher.NodeSidecars.failures() -> typing.List[str] ``` One message per sidecar that is no longer running. ```python nemo_gym.h2_ping_sidecar.launcher.NodeSidecars.start() -> typing.List[str] ``` Launch every instance and wait until each has bound its port; returns one summary line each. ```python nemo_gym.h2_ping_sidecar.launcher.NodeSidecars.stop() -> None ``` SIGTERM every sidecar (it drains in-flight requests), then SIGKILL stragglers. ```python class nemo_gym.h2_ping_sidecar.launcher.RewrittenUrl( location: str, old: str, new: str ) ``` Dataclass **`location`** `str` --- **`new`** `str` --- **`old`** `str` --- ```python class nemo_gym.h2_ping_sidecar.launcher.SidecarError() ``` **Bases:** `RuntimeError` The sidecar could not be configured, started or kept running. ```python nemo_gym.h2_ping_sidecar.launcher._local_go_version() -> str ``` `go env GOVERSION` of the Go on PATH, or an empty string when there is none. ```python nemo_gym.h2_ping_sidecar.launcher._local_url( url: str, instances_by_origin: typing.Dict[str, nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig], location: str ) -> typing.Optional[str] ``` ```python nemo_gym.h2_ping_sidecar.launcher._log_tail( log_path: str ) -> str ``` ```python nemo_gym.h2_ping_sidecar.launcher._source_digest() -> str ``` Short hash of the Go sources the binary is built from. A binary built from older sources must never be reused, so the hash is part of the cache path. ```python nemo_gym.h2_ping_sidecar.launcher.build_sidecar( output_path: pathlib.Path ) -> None ``` Compile the bundled Go source into `output_path`. The compiler writes to a private temporary file that is moved into place afterwards, so a concurrent run sharing the cache never executes a half-written binary. ```python nemo_gym.h2_ping_sidecar.launcher.default_binary_path( global_config_dict: typing.Any ) -> pathlib.Path ``` Where the sidecar built from the current sources lives: `<cache>/<sources hash>/<go version>/`. The proxy is mostly Go's own `net/http` and `crypto/tls`, whose fixes ship as Go releases, so a binary built by a Go that has since been replaced is not reused: with Go on PATH the exact version is part of the path. Without Go nothing can be built, so any binary already built from these sources is used (the newest one), which lets a node without Go run a prebuilt cache. ```python nemo_gym.h2_ping_sidecar.launcher.h2_ping_sidecar_config_from_global_config( global_config_dict: typing.Any ) -> nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig ``` Build the config from the optional top-level `sidecar:` block; absent means disabled. ```python nemo_gym.h2_ping_sidecar.launcher.resolve_binary( config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig, global_config_dict: typing.Any ) -> pathlib.Path ``` The sidecar executable, building it first when allowed and missing. ```python nemo_gym.h2_ping_sidecar.launcher.resolve_log_dir( config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig, global_config_dict: typing.Any ) -> str ``` ```python nemo_gym.h2_ping_sidecar.launcher.rewrite_base_urls( global_config_dict: omegaconf.DictConfig, instances: typing.List[nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig] ) -> typing.List[nemo_gym.h2_ping_sidecar.launcher.RewrittenUrl] ``` Point every model URL aimed at an instance's upstream at that instance's local address, in place. Covers any `*base_url` key (a string or a list of strings) of a model server under `responses_api_models`; top-level keys such as `policy_base_url` are left as they are. The *resolved* value is what gets matched, so a URL that comes from a resolver such as `${oc.env:JUDGE_URL}` or from an alias such as `${policy_base_url}` is routed through the sidecar too, and is materialized as a literal in memory for that field only. A field that aliases a whole list is rewritten as a resolved copy, so the list it points at is never changed. Raises `SidecarError` for a value that is not a parseable URL. ```python nemo_gym.h2_ping_sidecar.launcher.sidecar_command( binary: str, config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig, instance: nemo_gym.h2_ping_sidecar.config.SidecarInstanceConfig, ready_path: str ) -> typing.List[str] ``` ```python nemo_gym.h2_ping_sidecar.launcher.sidecar_env( config: nemo_gym.h2_ping_sidecar.config.H2PingSidecarConfig ) -> typing.Dict[str, str] ``` Environment for the child: ours, plus the Go runtime limits from the config. ```python nemo_gym.h2_ping_sidecar.launcher.start_h2_ping_sidecar( global_config_dict: omegaconf.DictConfig ) -> typing.Optional[nemo_gym.h2_ping_sidecar.launcher.H2PingSidecarManager] ``` Start the sidecar when `sidecar.enabled` and point model URLs at it. Returns the manager to poll and stop later, or None when the feature is off. Raises `SidecarError` (after stopping anything it started) when it cannot be brought up. Call it after `initialize_ray()` and before the config is serialized for the servers. Everything that can be checked without side effects is checked first, so a bad config fails with no process started and the live config untouched. Once the proxy is running, any later failure or interrupt stops it before the error propagates, because nothing else owns it yet. ```python nemo_gym.h2_ping_sidecar.launcher.SIDECAR_BINARY_NAME = 'h2-ping-sidecar' ``` ```python nemo_gym.h2_ping_sidecar.launcher.SIDECAR_SOURCE_DIR = Path(__file__).resolve().parent.parent / 'tools' / 'sidecar' ``` ```python nemo_gym.h2_ping_sidecar.launcher._BASE_URL_KEY_SUFFIX = 'base_url' ``` ```python nemo_gym.h2_ping_sidecar.launcher._BUILD_TIMEOUT_SEC = 300.0 ``` ```python nemo_gym.h2_ping_sidecar.launcher._KILL_REAP_TIMEOUT_SEC = 5.0 ``` ```python nemo_gym.h2_ping_sidecar.launcher._LOG_TAIL_LINES = 20 ``` ```python nemo_gym.h2_ping_sidecar.launcher._MODEL_SERVER_TYPE = 'responses_api_models' ``` ```python nemo_gym.h2_ping_sidecar.launcher._READY_POLL_INTERVAL_SEC = 0.1 ``` ```python nemo_gym.h2_ping_sidecar.launcher._STOP_SLACK_SEC = 5.0 ```