nemo_rl.weight_sync.xferdtensor#
XferDTensor: cross-mesh DTensor reshard for disaggregated refit.
Module Contents#
Classes#
DTensor-compatible reference for xferdtensor. |
Functions#
Whether |
|
Use Python only implementation in for the xferdtensor. |
|
Whether this process transfers with |
|
Make this refit destination transfer the way its source does. |
|
Public XferDTensor entry point used by all external callers. |
|
Return the slice of the global tensor this rank owns. |
|
Broadcast-based reference implementation of XferDTensor. |
Data#
API#
- nemo_rl.weight_sync.xferdtensor._XFERDTENSOR_PATH_LOGGED#
False
- class nemo_rl.weight_sync.xferdtensor.DTensorRef(
- local_tensor: torch.Tensor,
- global_shape,
- dtype=None,
- device=None,
DTensor-compatible reference for xferdtensor.
Provides the interface xferdtensor reads via duck typing:
.shape: global tensor shape (torch.Size)._local_tensor: local shard (src side) or dst buffer (dst side).dtype,.device: tensor metadata
On the src side (train),
local_tensoris the TP-local shard from Megatron parameters (no PP broadcast or TP gather needed).global_shapeis the full unsharded shape.On the dst side (gen),
local_tensoris either the vLLM local parameter (for direct params) or a temporary buffer (for merged/unmapped params).global_shapeis always the full unsharded shape.Initialization
- nemo_rl.weight_sync.xferdtensor._use_golden_api() bool#
Whether
NRL_XFERDTENSOR_GOLDENforces the golden reshard path.
- nemo_rl.weight_sync.xferdtensor._use_python_api() bool#
Use Python only implementation in for the xferdtensor.
xferdtensor_python_implis the backup implementation when the nccl.m2n.reshard is not available.
- nemo_rl.weight_sync.xferdtensor.uses_real_reshard_op() bool#
Whether this process transfers with
nccl.m2n.reshard.
- nemo_rl.weight_sync.xferdtensor.follow_source_reshard_path(source_uses_real_op: bool) None#
Make this refit destination transfer the way its source does.
The two ends run in separate venvs, so
nccl.m2ncan be installed on one end only. Mixing the real op with the Python transfer deadlocks in NCCL.
- nemo_rl.weight_sync.xferdtensor.xferdtensor(
- src_tensor,
- src_mesh,
- src_placement,
- dst_tensor,
- dst_mesh,
- dst_placement,
- process_group,
- stream=None,
Public XferDTensor entry point used by all external callers.
- nemo_rl.weight_sync.xferdtensor._flatten_mesh_ranks(mesh)#
- nemo_rl.weight_sync.xferdtensor._get_tensor_meta(src_tensor, dst_tensor)#
- nemo_rl.weight_sync.xferdtensor._get_mesh_coords(mesh, rank)#
- nemo_rl.weight_sync.xferdtensor._compute_shard_slices(
- global_shape,
- mesh_shape,
- mesh_coords,
- placements,
Return the slice of the global tensor this rank owns.
- nemo_rl.weight_sync.xferdtensor.xferdtensor_golden(
- src_tensor,
- src_mesh,
- src_placement,
- dst_tensor,
- dst_mesh,
- dst_placement,
- process_group,
Broadcast-based reference implementation of XferDTensor.
Reconstructs the full (global) tensor from TP-local shards on the source mesh, then each destination rank extracts its local shard based on
dst_placement.When all
src_placemententries areReplicate, a single broadcast fromsrc_ranks[0]suffices. Otherwise, one broadcast per unique shard region reconstructs the full tensor.This is the canonical 7-argument signature matching the real
nccl.m2n.reshardop. Callers must wrap raw tensors inDTensorRefso that.shapereports the global shape and._local_tensorholds the local shard.