nemo_rl.models.value.utils#

Shared utilities for aligning value-model predictions.

Module Contents#

Functions#

right_shift_values

Align next-token value-head outputs with their input states.

gather_and_right_shift_values

Restore global CP sequence order before value temporal alignment.

API#

nemo_rl.models.value.utils.right_shift_values(values: torch.Tensor) → torch.Tensor#

Align next-token value-head outputs with their input states.

The first column becomes zero and column t (for t >= 1) takes the prediction from column t - 1. The output shape matches the input shape.

nemo_rl.models.value.utils.gather_and_right_shift_values(
values: torch.Tensor,
cp_group: Optional[torch.distributed.ProcessGroup] = None,
) → torch.Tensor#

Restore global CP sequence order before value temporal alignment.