> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.core.serve.ray_serve.backend

## Module Contents

### Classes

| Name                                                                            | Description                              |
| ------------------------------------------------------------------------------- | ---------------------------------------- |
| [`RayServeBackend`](#nemo_curator-core-serve-ray_serve-backend-RayServeBackend) | Ray Serve backend for `InferenceServer`. |

### API

```python
class nemo_curator.core.serve.ray_serve.backend.RayServeBackend(
    server: nemo_curator.core.serve.server.InferenceServer
)
```

**Bases:** [InferenceBackend](/nemo-curator/nemo_curator/core/serve/base#nemo_curator-core-serve-base-InferenceBackend)

Ray Serve backend for `InferenceServer`.

```python
nemo_curator.core.serve.ray_serve.backend.RayServeBackend._cleanup_failed_deploy() -> None
```

staticmethod

Best-effort cleanup after a failed deploy.

```python
nemo_curator.core.serve.ray_serve.backend.RayServeBackend._deploy() -> None
```

Deploy models onto the connected Ray cluster.

```python
nemo_curator.core.serve.ray_serve.backend.RayServeBackend._quiet_runtime_env() -> dict[str, typing.Any]
```

staticmethod

Return a `runtime_env` dict that suppresses per-request logs.

```python
nemo_curator.core.serve.ray_serve.backend.RayServeBackend._reset_serve_client_cache() -> None
```

staticmethod

Reset Ray Serve's cached controller client.

TODO: Remove this once [https://github.com/ray-project/ray/issues/61608](https://github.com/ray-project/ray/issues/61608) is fixed.

```python
nemo_curator.core.serve.ray_serve.backend.RayServeBackend._to_llm_config(
    model: nemo_curator.core.serve.ray_serve.config.RayServeModelConfig,
    quiet_runtime_env: dict[str, typing.Any] | None = None
) -> ray.serve.llm.LLMConfig
```

staticmethod

Translate a typed Ray Serve model config into `LLMConfig`.

```python
nemo_curator.core.serve.ray_serve.backend.RayServeBackend.start() -> None
```

Connect to Ray, deploy the models, and detach the driver.

```python
nemo_curator.core.serve.ray_serve.backend.RayServeBackend.stop() -> None
```

Reconnect to Ray and tear down Ray Serve.