`bridge.data.builders.energon`#

Serializable configuration and runtime builder for Energon VLM data.

Module Contents#

Classes#

`EnergonTaskEncoderConfig`	Declarative extension contract for Energon task encoders.
`HFEnergonTaskEncoderConfig`	Serializable settings for the generic Hugging Face Energon task encoder.
`QwenVLEnergonTaskEncoderConfig`	Serializable settings for the Qwen-VL Energon task encoder.
`NemotronOmniEnergonTaskEncoderConfig`	Serializable settings for the Nemotron Omni Energon task encoder.
`EnergonDatasetConfig`	Serializable configuration for an Energon-backed multimodal dataset.
`EnergonDatasetBuilder`	Build Energon runtime dataloader iterators from declarative config.

Functions#

`_validate_hf_path`
`build_energon_task_encoder`	Construct the configured Energon task encoder at runtime.
`_build_builtin_energon_task_encoder`
`energon_train_valid_test_datasets_provider`	Build Energon iterators through the canonical runtime builder.

API#

bridge.data.builders.energon._validate_hf_path(path: str, *, field_name: str) → None#

class bridge.data.builders.energon.EnergonTaskEncoderConfig#

Declarative extension contract for Energon task encoders.

Subclass this config (or a built-in config) with model-owned dataclass fields and override build_task_encoder to construct the runtime encoder. Config instances must not store processors, tokenizers, or runtime encoders. Restoring custom config classes uses Bridge’s existing target allowlist.

supports_native_packing: ClassVar[bool]#

False

Whether the encoder implements Energon’s native packing contract.

validate() → None#: Validate model-owned fields without constructing runtime objects.

validate_dataset( dataset_config: bridge.data.builders.energon.EnergonDatasetConfig, ) → None#

Validate or synchronize model-owned fields with resolved dataset settings.

Called during dataset finalization and before runtime construction, after generic validation. Training finalizes the dataset after deriving padding settings; preliminary generic validation does not call this hook. Implementations must be idempotent, must not construct runtime objects, and must not modify generic dataset settings. Recipe/CLI overrides and training-derived padding settings are available when the builder runs.

Parameters:: dataset_config – Enclosing dataset settings used to validate or derive model-owned fields.

build_task_encoder( dataset_config: bridge.data.builders.energon.EnergonDatasetConfig, ) → object#

Construct an encoder from validated, resolved settings at runtime.

Custom encoders override this method and own loading their runtime assets. The default implementation constructs the built-in encoders.

Parameters:: dataset_config – Validated dataset settings after overrides.
Returns:: The runtime Energon task encoder.

class bridge.data.builders.energon.HFEnergonTaskEncoderConfig#

Bases: bridge.data.builders.energon.EnergonTaskEncoderConfig

Serializable settings for the generic Hugging Face Energon task encoder.

hf_processor_revision optionally pins the processor artifact load. visual_keys names processor output tensors retained in the model batch. min_pixels and max_pixels are independent processor preprocessing bounds controlling visual resolution and token cost; they are not output keys.

hf_processor_path: str#: None

hf_processor_revision: str | None#: None

visual_keys: tuple[str, ...]#: (‘pixel_values’,)

min_pixels: int | None#: None

max_pixels: int | None#: None

trust_remote_code: bool | None#: None

validate() → None#: Validate generic Hugging Face task-encoder settings.

class bridge.data.builders.energon.QwenVLEnergonTaskEncoderConfig#

Bases: bridge.data.builders.energon.EnergonTaskEncoderConfig

Serializable settings for the Qwen-VL Energon task encoder.

Qwen’s visual output keys are model-owned. min_pixels and max_pixels instead bound processor preprocessing and visual-token cost. hf_processor_revision optionally pins both tokenizer and processor artifact loads.

supports_native_packing: ClassVar[bool]#: True

hf_processor_path: str#: None

hf_processor_revision: str | None#: None

temporal_patch_size: int#: 2

spatial_merge_size: int#: 2

patch_size: int#: 14

min_pixels: int#: 200704

max_pixels: int#: 1003520

max_num_images: int | None#: 10

max_num_frames: int | None#: 60

max_visual_tokens: int | None#: 16384

trust_remote_code: bool | None#: None

validate() → None#: Validate Qwen-VL task-encoder settings.

class bridge.data.builders.energon.NemotronOmniEnergonTaskEncoderConfig#

Bases: bridge.data.builders.energon.EnergonTaskEncoderConfig

Serializable settings for the Nemotron Omni Energon task encoder.

visual_keys is retained for configuration compatibility, but Omni owns its visual input contract and supports only ("pixel_values",). hf_processor_revision optionally pins the processor artifact load. temporal_video_resize_mode="processor" is the default and applies the public aspect-preserving video grid with exact per-tubelet token counts. "fixed_512" preserves the previous square policy for compatibility. collapse_image_tokens=True selects the deprecated LLaVA compatibility path and cannot be combined with processor-driven temporal sizing. The default False selects the canonical expanded-sequence path.

hf_processor_path: str#: None

hf_processor_revision: str | None#: None

max_audio_duration: float#: None

num_mel_bins: int#: None

visual_keys: tuple[str, ...]#: None

temporal_patch_size: int#: None

video_fps: float#: None

video_nframes: int#: None

use_temporal_video_embedder: bool#: None

patch_dim: int#: None

temporal_video_resize_mode: Literal[fixed_512, processor]#: ‘processor’

collapse_image_tokens: bool#: False

trust_remote_code: bool | None#: None

validate() → None#: Validate Nemotron Omni task-encoder settings.

class bridge.data.builders.energon.EnergonDatasetConfig#

Bases: megatron.bridge.data.base.DataloaderConfig

Serializable configuration for an Energon-backed multimodal dataset.

path: str | None#: None

seq_length: int#: None

micro_batch_size: int#: None

task_encoder: bridge.data.builders.energon.EnergonTaskEncoderConfig#: None

dataloader_type: Literal[external] | None#: ‘external’

do_validation: bool#: True

do_test: bool#: False

num_val_workers: int | None#: None

shuffle_buffer_size: int#: 100

max_samples_per_sequence: int | None#: None

packing_buffer_size: int | None#: None

dataset_kwargs: dict[str, Any]#: ‘field(…)’

enable_in_batch_packing: bool#: False

defer_in_batch_packing_to_step: bool#: False

pad_to_max_length: bool#: False

pad_to_multiple_of: int#: 128

in_batch_packing_pad_to_multiple_of: int#: 1

validate() → None#: Validate declarative fields before training derives dataset settings.

finalize() → None#: Finalize dataloader fields and validate the resolved encoder settings.

bridge.data.builders.energon.build_energon_task_encoder( config: bridge.data.builders.energon.EnergonDatasetConfig, ) → Any#: Construct the configured Energon task encoder at runtime.

bridge.data.builders.energon._build_builtin_energon_task_encoder( config: bridge.data.builders.energon.EnergonDatasetConfig, ) → Any#

class bridge.data.builders.energon.EnergonDatasetBuilder( config: bridge.data.builders.energon.EnergonDatasetConfig, )#

Build Energon runtime dataloader iterators from declarative config.

Initialization

build( context: megatron.bridge.data.base.DatasetBuildContext, ) → tuple[Any | None, Any | None, None]#: Build requested Energon train and validation iterators.

bridge.data.builders.energon.energon_train_valid_test_datasets_provider( train_val_test_num_samples: list[int], dataset_config: bridge.data.builders.energon.EnergonDatasetConfig, tokenizer: Any | None = None, pg_collection: Any | None = None, ) → tuple[Any | None, Any | None, None]#: Build Energon iterators through the canonical runtime builder.

bridge.data.builders.energon#

Module Contents#

Classes#

Functions#

API#

`bridge.data.builders.energon`#