`nemo_rl.data.energon.multimodal.task_encoders.generic_sft`#

Module Contents#

Classes#

`SFTProcessorAdapter`	Boundary between canonical and model-specific SFT data.
`HFMultimodalSFTProcessorAdapter`	Hugging Face implementation of the generic processor boundary.
`GenericSFTTaskEncoder`	Encode, group, and batch complete multimodal SFT conversations.

Functions#

`_normalize_messages`	Validate the message structure and attach each part’s media.
`build_processor_adapter`	Build the configured model processor adapter.

Data#

__all__

API#

class nemo_rl.data.energon.multimodal.task_encoders.generic_sft.SFTProcessorAdapter#

Bases: typing.Protocol

Boundary between canonical and model-specific SFT data.

property fingerprint: str#

encode( sample: nemo_rl.data.energon.multimodal.types.CanonicalSFTSample, ) → nemo_rl.data.energon.multimodal.types.EncodedSFTSample#

nemo_rl.data.energon.multimodal.task_encoders.generic_sft._normalize_messages( sample: nemo_rl.data.energon.multimodal.types.CanonicalSFTSample, *, materialize: bool = True, ) → list[dict[str, Any]]#

Validate the message structure and attach each part’s media.

Parameters:

sample – The cooked conversation.
materialize – Decode each media value and attach the payload. Set False to attach the MediaRef instead.

The Nemotron renderers replace every media part with text built from metadata and then overwrite message["content"] wholesale, so decoding for them is pure waste. It is also waste paid at the wrong time: this runs in pre-encode, before select_samples_to_pack, so rows that selection discards are decoded too. Measured on video rows at 2771 ms against the Megatron reference’s 4.7 ms, which defers all frame work to post-encode.

Only GenericSFTTaskEncoder.encode consumes the payload, via get_formatted_message_log, so it keeps the default.

class nemo_rl.data.energon.multimodal.task_encoders.generic_sft.HFMultimodalSFTProcessorAdapter( *, processor: Any, max_sequence_length: int, add_bos: bool, add_eos: bool, add_generation_prompt: bool, )#

Hugging Face implementation of the generic processor boundary.

Initialization

property fingerprint: str#

encode( sample: nemo_rl.data.energon.multimodal.types.CanonicalSFTSample, ) → nemo_rl.data.energon.multimodal.types.EncodedSFTSample#

class nemo_rl.data.energon.multimodal.task_encoders.generic_sft.GenericSFTTaskEncoder( *, adapter: nemo_rl.data.energon.multimodal.task_encoders.generic_sft.SFTProcessorAdapter, cooker_functions: collections.abc.Sequence[nemo_rl.data.energon.multimodal.task_encoders.base.SFTCooker], include_source_ids: bool, packer: nemo_rl.data.packing.SequencePacker | None = None, tokenizer: Any | None = None, sequence_length_pad_multiple: int = 1, only_unmask_final: bool = False, )#

Bases: nemo_rl.data.energon.multimodal.task_encoders.base.BaseSFTTaskEncoder

Encode, group, and batch complete multimodal SFT conversations.