bridge.models.nemotron_omni.nemotron_omni_bridge#

Nemotron Omni bridge.

Standalone bridge for the Nemotron-3 Omni family (HF architecture NemotronH_Nano_Omni_Reasoning_V3). Inherits the language / vision / mamba parameter mappings from :class:NemotronVLBridge and adds:

  • Omni-specific CONFIG_MAPPING entries (Mamba shape fields used by the hybrid LLM and the MoE shared-expert intermediate size).

  • An overridden :meth:provider_bridge that produces a

    class:

    NemotronOmniModelProvider (MoE language model + RADIO ViT vision

    • optional Parakeet sound encoder) instead of the dense VL provider.

  • A :meth:mapping_registry override that adds the temporal video_embedder parameter and the sound projection / sound encoder parameters (the latter via a single ** wildcard, since the Megatron sound encoder is HF transformers’ ParakeetEncoder and the parameter names line up 1:1 with sound_encoder.encoder.*).

  • ADDITIONAL_FILE_PATTERNS covering the bespoke Omni HF modeling / processing / audio files that need to be copied during HF export.

Module Contents#

Classes#

NemotronOmniBridge

Bridge for Nemotron-3 Omni (MoE LLM + vision + optional sound) models.

API#

class bridge.models.nemotron_omni.nemotron_omni_bridge.NemotronOmniBridge#

Bases: megatron.bridge.models.nemotron_vl.nemotron_vl_bridge.NemotronVLBridge

Bridge for Nemotron-3 Omni (MoE LLM + vision + optional sound) models.

CONFIG_MAPPING#

None

ADDITIONAL_FILE_PATTERNS#

[‘modeling*.py’, ‘configuration*.py’, ‘processing*.py’, ‘processing_utils.py’, ‘image_processing*.py…

provider_bridge(
hf_pretrained: megatron.bridge.models.hf_pretrained.vlm.PreTrainedVLM,
) → megatron.bridge.models.nemotron_omni.nemotron_omni_provider.NemotronOmniModelProvider#

Create a NemotronOmniModelProvider from the HF Omni config.

Always returns an Omni provider (MoE language model + RADIO ViT vision + optional Parakeet sound encoder). When sound_config is absent on the HF config, has_sound=False and the sound branch is skipped at construction time.

mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry#

Inherit VL mappings and add temporal video, sound projection, and sound encoder mappings.