bridge.models.nemotron_omni.modeling_nemotron_omni_llava#

Module Contents#

Classes#

NemotronOmniLlavaModel

Deprecated collapse/expand Omni wrapper around MCore LLaVAModel.

API#

class bridge.models.nemotron_omni.modeling_nemotron_omni_llava.NemotronOmniLlavaModel(
config: megatron.bridge.models.hybrid.hybrid_provider.HybridModelProvider | None = None,
*,
llava_model: megatron.core.models.multimodal.llava_model.LLaVAModel | None = None,
pre_process: bool | None = True,
post_process: bool | None = True,
vp_stage: int | None = None,
)#

Bases: megatron.bridge.models.nemotron_vl.modeling_nemotron_vl.NemotronVLModel

Deprecated collapse/expand Omni wrapper around MCore LLaVAModel.

forward() normalizes legacy image metadata and then delegates to NemotronVLModel/LLaVAModel.

Use :class:~megatron.bridge.models.nemotron_omni.modeling_nemotron_omni.NemotronOmniModel for the canonical processor-expanded sequence and collator-owned packing contract.

Initialization

Construct the deprecated LLaVA compatibility model.

Parameters:

  • config – Provider used to construct the wrapped model.

  • llava_model – Fully assembled MCore LLaVA model.

  • pre_process – Whether this pipeline stage owns input processing.

  • post_process – Whether this pipeline stage owns output processing.

  • vp_stage – Optional virtual pipeline stage.

forward(*args, num_frames: torch.Tensor | int | None = None, **kwargs)#

Delegate to MCore after normalizing the image-only frame contract.

Parameters:

  • *args – Positional arguments forwarded to MCore LLaVAModel.

  • num_frames – Per-image frame counts. The deprecated LLaVA path only supports images, so every count must be one.

  • **kwargs – Keyword arguments forwarded to MCore LLaVAModel.

Returns:

The wrapped MCore model output.

Raises:

NotImplementedError – If any input contains more than one frame.

freeze(
*,
freeze_language_model: bool = False,
freeze_vision_model: bool = False,
freeze_vision_projection: bool = False,
freeze_sound_model: bool = False,
freeze_sound_projection: bool = False,
) → None#