nemo_curator.core.serve.ray_serve.config

Module Contents

Classes

Name	Description
`RayServeModelConfig`	Ray Serve model config.
`RayServeServerConfig`	Server-level Ray Serve config.

API

class nemo_curator.core.serve.ray_serve.config.RayServeModelConfig(
    model_identifier: str,
    model_name: str | None = None,
    runtime_env: dict[str, typing.Any] = dict(),
    deployment_config: dict[str, typing.Any] = dict(),
    engine_kwargs: dict[str, typing.Any] = dict()
)

Dataclass

Bases: BaseModelConfig

Ray Serve model config.

deployment_config

dict[str, Any] = field(default_factory=dict)

engine_kwargs

dict[str, Any] = field(default_factory=dict)

class nemo_curator.core.serve.ray_serve.config.RayServeServerConfig()

Dataclass

Bases: BaseServerConfig

Server-level Ray Serve config.

model_configs

tuple[type[BaseModelConfig], ...] = (RayServeModelConfig,)