`nemo_rl.models.generation.dynamo.engine_adapter`#

Serve a NeMo RL generation worker’s own engine through Dynamo’s backend SDK.

DynamoEngineAdapter is the Dynamo side, written once for every backend: the request kinds served, how errors reach the driver, and the endpoint’s registration. A backend implements four methods over its worker’s engine (serve_rollout, start, abort and kv_event_sources); vllm_adapter.DynamoVllmAdapter is the vLLM one. The engine only ever receives token IDs, on /v1/completions. This module imports no generation backend.

Module Contents#

Classes#

`ServedRollout`	A backend’s answer to one rollout.
`DynamoEngineAdapter`	Serves one NeMo RL generation worker through Dynamo. The worker owns the engine.

Functions#

`_keep_host_logging`	Replaces Dynamo’s `configure_dynamo_logging` (ai-dynamo/dynamo#18447).
`_annotation_value`	The value of the first `<prefix><value>` annotation, or None.
`start_dynamo_endpoint`	Register `adapter`’s engine with Dynamo and serve it on the running loop.

Data#

RequestHandler

API#

nemo_rl.models.generation.dynamo.engine_adapter.RequestHandler#: None

class nemo_rl.models.generation.dynamo.engine_adapter.ServedRollout#

A backend’s answer to one rollout.

result: nemo_rl.models.generation.rollout_result.RolloutResult#: None

finish_reason: str#: None

usage: Optional[dict[str, Any]]#: None

nemo_rl.models.generation.dynamo.engine_adapter._keep_host_logging(*args: object, **kwargs: object) → None#

Replaces Dynamo’s configure_dynamo_logging (ai-dynamo/dynamo#18447).

Worker.run() calls it first, and it would replace this process’s Python logging: NeMo RL’s root handler and level, and vLLM’s handlers.

nemo_rl.models.generation.dynamo.engine_adapter._annotation_value( request: dynamo.common.backend.GenerateRequest, prefix: str, ) → Optional[str]#: The value of the first <prefix><value> annotation, or None.

class nemo_rl.models.generation.dynamo.engine_adapter.DynamoEngineAdapter(*, leader_worker_idx: int)#

Bases: dynamo.common.backend.LLMEngine

Serves one NeMo RL generation worker through Dynamo. The worker owns the engine.

Initialization

abstractmethod async classmethod from_args(argv=None)#

_handlers() → dict[str, nemo_rl.models.generation.dynamo.engine_adapter.RequestHandler]#: The request kinds served, keyed by the annotation prefix that marks them.

async generate( request: dynamo.common.backend.GenerateRequest, context: dynamo._core.Context, ) → collections.abc.AsyncGenerator[dynamo.common.backend.GenerateChunk, None]#

async _generate_rollout( request: dynamo.common.backend.GenerateRequest, rollout_spec: str, context: dynamo._core.Context, ) → collections.abc.AsyncGenerator[dynamo.common.backend.GenerateChunk, None]#

One NeMo RL rollout, served by the backend’s serve_rollout.

One chunk carries the result, routed experts included, in engine_data; the driver assembles the same output generate_async returns.

abstractmethod async serve_rollout( prompt_token_ids: list[int], *, greedy: bool, stop_strings: Optional[list[str]], request_id: str, ) → nemo_rl.models.generation.dynamo.engine_adapter.ServedRollout#: Serve one rollout exactly as the worker’s native generate_async does.

abstractmethod async start(worker_id: int) → dynamo.common.backend.EngineConfig#: Register the engine’s context length, KV block size and capacity.

abstractmethod async abort(context: dynamo._core.Context) → None#: Cancel the request context names; Dynamo calls it on disconnect.

abstractmethod async kv_event_sources() → list[dynamo.common.backend.publisher.KvEventSource]#: Where the engine publishes the KV-cache events Dynamo’s KV router indexes.

async cleanup() → None#: Nothing to release: the worker owns its engine and shuts it down.

nemo_rl.models.generation.dynamo.engine_adapter.start_dynamo_endpoint( adapter: nemo_rl.models.generation.dynamo.engine_adapter.DynamoEngineAdapter, *, model_path: str, served_model_name: str, namespace: str, engine_env: dict[str, str], ) → asyncio.Task#

Register adapter’s engine with Dynamo and serve it on the running loop.

model_path is a local directory with the model’s config and tokenizer files; the engine is served under served_model_name.

The Rust Worker serves until the process gets SIGTERM. It installs a chained handler, so Ray’s actor handler still runs and the actor exits as usual; discovery drops the engine with its process.

nemo_rl.models.generation.dynamo.engine_adapter#