> This page is for version 26.07 · v1.3.0.
> For other versions, use one of these documentation indexes:
> - Latest · v1.4.0 (26.09) (default): https://docs.nvidia.com/nemo/curator/latest/llms.txt
> - Main · preview: https://docs.nvidia.com/nemo/curator/main/llms.txt
> - 26.09 · v1.4.0: https://docs.nvidia.com/nemo/curator/v26.09/llms.txt
> - 26.07 · v1.3.0: https://docs.nvidia.com/nemo/curator/v26.07/llms.txt
> - 26.04 · v1.2.0: https://docs.nvidia.com/nemo/curator/v26.04/llms.txt
> - 26.02 · v1.1.0: https://docs.nvidia.com/nemo/curator/v26.02/llms.txt
> - 25.09 · v1.0.0: https://docs.nvidia.com/nemo/curator/v25.09/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.backends.ray_data.diagnostics

Runtime Ray Data scheduler diagnostics for the supported Ray release.

The diagnostics were developed as a Ray source patch, but Curator cannot
distribute a patched Ray wheel.  This module installs the equivalent Python
hooks in the driver process.  The hooks emit through child loggers of
`ray.data` so Ray's `SessionFileHandler` writes them to
`session_latest/logs/ray-data/ray-data.log`.

All affected scheduler components run in the Ray Data driver.  Worker
environments therefore do not need modified Ray installations.

## Module Contents

### Classes

| Name                                                                                               | Description                                          |
| -------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| [`DiagnosticsInstallStatus`](#nemo_curator-backends-ray_data-diagnostics-DiagnosticsInstallStatus) | Result of attempting to enable Ray Data diagnostics. |
| [`_TaskAdmissionDecision`](#nemo_curator-backends-ray_data-diagnostics-_TaskAdmissionDecision)     | -                                                    |

### Functions

| Name                                                                                                                               | Description                                                             |
| ---------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| [`_has_native_diagnostics`](#nemo_curator-backends-ray_data-diagnostics-_has_native_diagnostics)                                   | -                                                                       |
| [`_install_actor_autoscaling_diagnostics`](#nemo_curator-backends-ray_data-diagnostics-_install_actor_autoscaling_diagnostics)     | -                                                                       |
| [`_install_downstream_capacity_diagnostics`](#nemo_curator-backends-ray_data-diagnostics-_install_downstream_capacity_diagnostics) | -                                                                       |
| [`_install_resource_admission_diagnostics`](#nemo_curator-backends-ray_data-diagnostics-_install_resource_admission_diagnostics)   | -                                                                       |
| [`_install_scheduling_reasons`](#nemo_curator-backends-ray_data-diagnostics-_install_scheduling_reasons)                           | -                                                                       |
| [`_milliseconds`](#nemo_curator-backends-ray_data-diagnostics-_milliseconds)                                                       | -                                                                       |
| [`_object_store_memory_fields`](#nemo_curator-backends-ray_data-diagnostics-_object_store_memory_fields)                           | -                                                                       |
| [`execution_resource_fields`](#nemo_curator-backends-ray_data-diagnostics-execution_resource_fields)                               | Flatten Ray `ExecutionResources` into stable scalar fields.             |
| [`format_logfmt_event`](#nemo_curator-backends-ray_data-diagnostics-format_logfmt_event)                                           | Format an event as stable, parseable logfmt-like tokens.                |
| [`install_ray_data_diagnostics`](#nemo_curator-backends-ray_data-diagnostics-install_ray_data_diagnostics)                         | Install driver-side diagnostics without modifying the Ray installation. |

### Data

[`RAY_DATA_DIAGNOSTICS_ENV_VAR`](#nemo_curator-backends-ray_data-diagnostics-RAY_DATA_DIAGNOSTICS_ENV_VAR)

[`_INSTALL_LOCK`](#nemo_curator-backends-ray_data-diagnostics-_INSTALL_LOCK)

[`_INSTALL_MARKER`](#nemo_curator-backends-ray_data-diagnostics-_INSTALL_MARKER)

[`_SUPPORTED_RAY_VERSION`](#nemo_curator-backends-ray_data-diagnostics-_SUPPORTED_RAY_VERSION)

[`_TRUE_ENV_VALUES`](#nemo_curator-backends-ray_data-diagnostics-_TRUE_ENV_VALUES)

### API

```python
class nemo_curator.backends.ray_data.diagnostics.DiagnosticsInstallStatus
```

**Bases:** `enum.Enum`

Result of attempting to enable Ray Data diagnostics.

**`ALREADY_INSTALLED`** `= 'already_installed'`

---

**`DISABLED`** `= 'disabled'`

---

**`INSTALLED`** `= 'installed'`

---

**`NATIVE`** `= 'native'`

---

**`UNSUPPORTED`** `= 'unsupported'`

---

```python
class nemo_curator.backends.ray_data.diagnostics._TaskAdmissionDecision(
    allowed: bool,
    reason: str,
    incremental_resources: typing.Any,
    remaining_budget: typing.Any,
    pending_output_estimate: float | None,
    op_usage: typing.Any = None,
    allocation: typing.Any = None
)
```

Dataclass

**`allowed`** `bool`

---

**`pending_output_estimate`** `float | None`

---

**`reason`** `str`

---

```python
nemo_curator.backends.ray_data.diagnostics._has_native_diagnostics(
    autoscaler_module: typing.Any,
    resource_manager_module: typing.Any,
    executor_state_module: typing.Any
) -> bool
```

```python
nemo_curator.backends.ray_data.diagnostics._install_actor_autoscaling_diagnostics(
    autoscaler_module: typing.Any
) -> None
```

```python
nemo_curator.backends.ray_data.diagnostics._install_downstream_capacity_diagnostics(
    downstream_policy_module: typing.Any
) -> None
```

```python
nemo_curator.backends.ray_data.diagnostics._install_resource_admission_diagnostics(
    resource_manager_module: typing.Any,
    resource_policy_module: typing.Any
) -> None
```

```python
nemo_curator.backends.ray_data.diagnostics._install_scheduling_reasons(
    executor_state_module: typing.Any
) -> None
```

```python
nemo_curator.backends.ray_data.diagnostics._milliseconds(
    seconds: float
) -> float
```

```python
nemo_curator.backends.ray_data.diagnostics._object_store_memory_fields(
    resource_manager: typing.Any,
    op: typing.Any
) -> dict[str, object]
```

```python
nemo_curator.backends.ray_data.diagnostics.execution_resource_fields(
    prefix: str,
    resources: typing.Any
) -> dict[str, object]
```

Flatten Ray `ExecutionResources` into stable scalar fields.

```python
nemo_curator.backends.ray_data.diagnostics.format_logfmt_event(
    event: str,
    fields: dict[str, object]
) -> str
```

Format an event as stable, parseable logfmt-like tokens.

```python
nemo_curator.backends.ray_data.diagnostics.install_ray_data_diagnostics() -> nemo_curator.backends.ray_data.diagnostics.DiagnosticsInstallStatus
```

Install driver-side diagnostics without modifying the Ray installation.

Diagnostics are opt-in through `NEMO_CURATOR_RAY_DATA_DIAGNOSTICS`.
The shim is intentionally restricted to the Ray version whose private APIs
it targets.  A future Ray release containing the upstream diagnostics is
detected and left untouched.

```python
nemo_curator.backends.ray_data.diagnostics.RAY_DATA_DIAGNOSTICS_ENV_VAR = 'NEMO_CURATOR_RAY_DATA_DIAGNOSTICS'
```

```python
nemo_curator.backends.ray_data.diagnostics._INSTALL_LOCK = threading.Lock()
```

```python
nemo_curator.backends.ray_data.diagnostics._INSTALL_MARKER = '_nemo_curator_ray_data_diagnostics_installed'
```

```python
nemo_curator.backends.ray_data.diagnostics._SUPPORTED_RAY_VERSION = '2.57.0'
```

```python
nemo_curator.backends.ray_data.diagnostics._TRUE_ENV_VALUES = {'1', 'true', 'yes', 'on'}
```