For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
DocumentationAPI Reference
DocumentationAPI Reference
  • API Reference
    • Overview
        • Nemo Curator
          • Backends
          • Config
          • Core
            • Client
            • Constants
            • Serve
              • Base
              • Constants
              • Dynamo
              • Placement
              • Ray Serve
                • Backend
                • Config
              • Server
              • Subprocess Mgr
            • Utils
          • Metrics
          • Models
          • Package Info
          • Pipeline
          • Stages
          • Tasks
          • Utils
    • Pipeline
    • ProcessingStage
    • CompositeStage
    • Resources
NVIDIANVIDIA
Developer-friendly docs for your API
Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.

LogoLogoNeMo Curator
On this page
  • Module Contents
  • Classes
  • API
API ReferenceFull Library ReferenceNemo CuratorNemo CuratorCoreServeRay Serve

nemo_curator.core.serve.ray_serve.backend

||View as Markdown|
Previous

nemo_curator.core.serve.ray_serve

Next

nemo_curator.core.serve.ray_serve.config

Module Contents

Classes

NameDescription
RayServeBackendRay Serve backend for InferenceServer.

API

class nemo_curator.core.serve.ray_serve.backend.RayServeBackend(
    server: nemo_curator.core.serve.server.InferenceServer
)

Bases: InferenceBackend

Ray Serve backend for InferenceServer.

nemo_curator.core.serve.ray_serve.backend.RayServeBackend._cleanup_failed_deploy() -> None
staticmethod

Best-effort cleanup after a failed deploy.

nemo_curator.core.serve.ray_serve.backend.RayServeBackend._deploy() -> None

Deploy models onto the connected Ray cluster.

nemo_curator.core.serve.ray_serve.backend.RayServeBackend._quiet_runtime_env() -> dict[str, typing.Any]
staticmethod

Return a runtime_env dict that suppresses per-request logs.

nemo_curator.core.serve.ray_serve.backend.RayServeBackend._reset_serve_client_cache() -> None
staticmethod

Reset Ray Serve’s cached controller client.

TODO: Remove this once https://github.com/ray-project/ray/issues/61608 is fixed.

nemo_curator.core.serve.ray_serve.backend.RayServeBackend._to_llm_config(
    model: nemo_curator.core.serve.ray_serve.config.RayServeModelConfig,
    quiet_runtime_env: dict[str, typing.Any] | None = None
) -> ray.serve.llm.LLMConfig
staticmethod

Translate a typed Ray Serve model config into LLMConfig.

nemo_curator.core.serve.ray_serve.backend.RayServeBackend.start() -> None

Connect to Ray, deploy the models, and detach the driver.

nemo_curator.core.serve.ray_serve.backend.RayServeBackend.stop() -> None

Reconnect to Ray and tear down Ray Serve.