> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.distributed.ddp

## Module Contents

### Classes

| Name                                                                  | Description                                           |
| --------------------------------------------------------------------- | ----------------------------------------------------- |
| [`DDPManager`](#nemo_automodel-components-distributed-ddp-DDPManager) | Manager for distributed training using PyTorch's DDP. |

### Data

[`logger`](#nemo_automodel-components-distributed-ddp-logger)

### API

```python
class nemo_automodel.components.distributed.ddp.DDPManager(
    config: nemo_automodel.components.distributed.config.DDPConfig
)
```

Manager for distributed training using PyTorch's DDP.

This manager wraps models with DistributedDataParallel for data-parallel
distributed training.

**Parameters:**

Configuration for DDP distributed training.

```python
nemo_automodel.components.distributed.ddp.DDPManager._setup_distributed()
```

Initialize device configuration for DDP.

Sets the rank, world\_size, and device based on the process group backend.

```python
nemo_automodel.components.distributed.ddp.DDPManager.parallelize(
    model
)
```

Wraps the given model with DistributedDataParallel (DDP).

Moves the model to the initialized device before wrapping. For CUDA devices,
the device id is passed to DDP as device\_ids; for CPU, no device ids are provided.

**Parameters:**

The PyTorch model to be wrapped.

**Returns:**

torch.nn.parallel.DistributedDataParallel: The DDP-wrapped model.

```python
nemo_automodel.components.distributed.ddp.logger = logging.getLogger(__name__)
```