`nemo_rl.utils.weight_transfer_http`#

HTTP control-plane utilities shared by remote sparse-refit transports.

Module Contents#

Functions#

`vllm_refit_endpoints`
`vllm_refit_api_key`
`refit_http_session`
`_http_executor`
`post_vllm_refit_endpoints`
`merge_vllm_refit_metrics`

Data#

`G_VLLM_REFIT_S3_MANIFEST_PATH`
`G_VLLM_REFIT_PREPARE_PATH`
`G_VLLM_REFIT_FLUSH_PATH`
`G_VLLM_REFIT_ZMQ_FLUSH_PATH`
`G_VLLM_REFIT_API_KEY_HEADER`
`_HTTP_LOCAL`
`_HTTP_ADAPTER`

API#

nemo_rl.utils.weight_transfer_http.G_VLLM_REFIT_S3_MANIFEST_PATH#: ‘/nemo-rl/refit/s3-manifest’

nemo_rl.utils.weight_transfer_http.G_VLLM_REFIT_PREPARE_PATH#: ‘/nemo-rl/refit/prepare’

nemo_rl.utils.weight_transfer_http.G_VLLM_REFIT_FLUSH_PATH#: ‘/nemo-rl/refit/flush’

nemo_rl.utils.weight_transfer_http.G_VLLM_REFIT_ZMQ_FLUSH_PATH#: ‘/nemo-rl/refit/zmq-flush’

nemo_rl.utils.weight_transfer_http.G_VLLM_REFIT_API_KEY_HEADER#: ‘x-nemo-rl-refit-key’

nemo_rl.utils.weight_transfer_http._HTTP_LOCAL#: ‘local(…)’

nemo_rl.utils.weight_transfer_http._HTTP_ADAPTER#: ‘HTTPAdapter(…)’

nemo_rl.utils.weight_transfer_http.vllm_refit_endpoints( base_urls: collections.abc.Sequence[str], path: str, ) → list[str][source]#

nemo_rl.utils.weight_transfer_http.vllm_refit_api_key(api_key_env_var: str | None) → str | None[source]#

nemo_rl.utils.weight_transfer_http.refit_http_session() → requests.Session[source]#

nemo_rl.utils.weight_transfer_http._http_executor(workers: int) → concurrent.futures.ThreadPoolExecutor[source]#

nemo_rl.utils.weight_transfer_http.post_vllm_refit_endpoints( endpoint_urls: collections.abc.Sequence[str], body: collections.abc.Mapping[str, Any], *, api_key: str | None, timeout_s: float, ) → list[dict[str, Any]][source]#

nemo_rl.utils.weight_transfer_http.merge_vllm_refit_metrics( result: dict[str, Any], metrics: collections.abc.Iterable[collections.abc.Mapping[str, Any]], *, maximum: bool, candidate_maximum: bool | None = None, ) → dict[str, Any][source]#