bridge.models.nemotron_omni.modeling_nemotron_omni_llava#

Module Contents#

Classes#

NemotronOmniLlavaModel

Deprecated collapse/expand Omni wrapper around MCore LLaVAModel.

API#

class bridge.models.nemotron_omni.modeling_nemotron_omni_llava.NemotronOmniLlavaModel(
config: megatron.bridge.models.nemotron_vl.nemotron_vl_provider.NemotronVLModelProvider | None = None,
*,
llava_model: megatron.core.models.multimodal.llava_model.LLaVAModel | None = None,
pre_process: bool | None = True,
post_process: bool | None = True,
vp_stage: int | None = None,
)#

Bases: megatron.bridge.models.nemotron_vl.modeling_nemotron_vl.NemotronVLModel

Deprecated collapse/expand Omni wrapper around MCore LLaVAModel.

forward() is inherited from NemotronVLModel (which delegates to LLaVAModel), so sound kwargs (sound_clips, sound_length) pass through automatically when the selected LLaVAModel implementation supports them.

Use :class:~megatron.bridge.models.nemotron_omni.modeling_nemotron_omni.NemotronOmniModel for the canonical processor-expanded sequence and collator-owned packing contract.

Initialization

Construct the deprecated LLaVA compatibility model.

Parameters:

  • config – Provider used to construct the wrapped model.

  • llava_model – Fully assembled MCore LLaVA model.

  • pre_process – Whether this pipeline stage owns input processing.

  • post_process – Whether this pipeline stage owns output processing.

  • vp_stage – Optional virtual pipeline stage.

freeze(
*,
freeze_language_model: bool = False,
freeze_vision_model: bool = False,
freeze_vision_projection: bool = False,
freeze_sound_model: bool = False,
freeze_sound_projection: bool = False,
) → None#