nemo_rl.models.value.utils#
Shared utilities for aligning value-model predictions.
Module Contents#
Functions#
Align next-token value-head outputs with their input states. |
|
Restore global CP sequence order before value temporal alignment. |
API#
- nemo_rl.models.value.utils.right_shift_values(values: torch.Tensor) torch.Tensor#
Align next-token value-head outputs with their input states.
The first column becomes zero and column
t(fort >= 1) takes the prediction from columnt - 1. The output shape matches the input shape.
- nemo_rl.models.value.utils.gather_and_right_shift_values(
- values: torch.Tensor,
- cp_group: Optional[torch.distributed.ProcessGroup] = None,
Restore global CP sequence order before value temporal alignment.