> This page is for version 0.5.0 · 26.06.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.distributed.ddp

## Module Contents

### Classes

| Name                                                                  | Description                                       |
| --------------------------------------------------------------------- | ------------------------------------------------- |
| [`DDPManager`](#nemo_automodel-components-distributed-ddp-DDPManager) | Deprecated compatibility wrapper for PyTorch DDP. |

### Functions

| Name                                                                                    | Description                                                          |
| --------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| [`_resolve_ddp_device`](#nemo_automodel-components-distributed-ddp-_resolve_ddp_device) | Validate distributed state and return the device owned by this rank. |
| [`parallelize_ddp`](#nemo_automodel-components-distributed-ddp-parallelize_ddp)         | Apply activation checkpointing and PyTorch DDP from a typed config.  |

### Data

[`logger`](#nemo_automodel-components-distributed-ddp-logger)

### API

```python
class nemo_automodel.components.distributed.ddp.DDPManager(
    config: nemo_automodel.components.distributed.config.DDPConfig
)
```

Deprecated compatibility wrapper for PyTorch DDP.

.. deprecated:: 0.7
Pass `DDPConfig` through the config-driven infrastructure and
provide model-specific behavior with `ModelParallelizer`. This
compatibility class is scheduled for removal in 0.8.

This manager wraps models with DistributedDataParallel for data-parallel
distributed training.

**Parameters:**

**`config`** `DDPConfig`

Configuration for DDP distributed training.

---

```python
nemo_automodel.components.distributed.ddp.DDPManager.__getattr__(
    name
)
```

```python
nemo_automodel.components.distributed.ddp.DDPManager._setup_distributed()
```

Initialize device configuration for DDP.

Sets the rank, world\_size, and device based on the process group backend.

```python
nemo_automodel.components.distributed.ddp.DDPManager.parallelize(
    model: torch.nn.Module,
    reapply_trainability: collections.abc.Callable[[torch.nn.Module], None] | None = None
) -> torch.nn.Module
```

Wraps the given model with DistributedDataParallel (DDP).

Moves the model to the initialized device before wrapping. For CUDA devices,
the device id is passed to DDP as device\_ids; for CPU, no device ids are provided.

**Parameters:**

**`model`** `torch.nn.Module`

The PyTorch model to be wrapped.

---

**`reapply_trainability`** `Callable[[torch.nn.Module], None] | None` — default: None

Optional callback that re-resolves parameter
trainability after model surgery and before DDP construction.

---

**Returns:** `torch.nn.Module`

torch.nn.parallel.DistributedDataParallel: The DDP-wrapped model.

```python
nemo_automodel.components.distributed.ddp._resolve_ddp_device() -> torch.device
```

Validate distributed state and return the device owned by this rank.

```python
nemo_automodel.components.distributed.ddp.parallelize_ddp(
    model: torch.nn.Module,
    config: nemo_automodel.components.distributed.config.DDPConfig,
    device: torch.device | None = None,
    reapply_trainability: collections.abc.Callable[[torch.nn.Module], None] | None = None
) -> torch.nn.Module
```

Apply activation checkpointing and PyTorch DDP from a typed config.

```python
nemo_automodel.components.distributed.ddp.logger = logging.getLogger(__name__)
```