`bridge.models.exaone.exaone45.modelling_exaone45.transformer_config`#

Module Contents#

Classes#

Exaone45TransformerConfig

Configuration for the EXAONE 4.5 transformer with vision and language components.

Functions#

get_vision_model_config

Get the vision model config for Exaone45 vision model.

API#

class bridge.models.exaone.exaone45.modelling_exaone45.transformer_config.Exaone45TransformerConfig#

Bases: megatron.core.transformer.transformer_config.TransformerConfig

Configuration for the EXAONE 4.5 transformer with vision and language components.

vocab_size: int#: 153600

language_max_sequence_length: int#: 131072

patch_size: int#: 16

temporal_patch_size: int#: 2

in_channels: int#: 3

spatial_merge_size: int#: 2

out_hidden_size: int#: 2048

apply_rotary_pos_emb_in_fp32: bool#: False

fullatt_block_indexes: List[int]#: ‘field(…)’

fp16_lm_cross_entropy: bool#: False

share_embeddings_and_output_weights: bool#: False

rotary_percent: float#: 1.0

apply_rope_fusion: bool#: False

image_token_id: int#: 67

video_token_id: int#: 68

vision_start_token_id: int#: 73

hf_text_config: Optional[transformers.models.exaone4_5.configuration_exaone4_5.Exaone4_5_Config]#: None

use_hf_vision_model: bool#: False

bridge.models.exaone.exaone45.modelling_exaone45.transformer_config.get_vision_model_config(hf_config, megatron_config=None)#: Get the vision model config for Exaone45 vision model.