bridge.models.nemotron_omni.nemotron_omni_bridge#

Nemotron Omni conversion bridges.

Standalone bridge for the Nemotron-3 Omni family (HF architecture NemotronH_Nano_Omni_Reasoning_V3). Inherits the language / vision / mamba parameter mappings from :class:NemotronVLBridge and adds:

  • Omni-specific CONFIG_MAPPING entries (Mamba shape fields used by the hybrid LLM and the MoE shared-expert intermediate size).

  • An overridden :meth:provider_bridge that produces a

    class:

    NemotronOmniModelProvider (MoE language model + RADIO ViT vision

    • optional Parakeet sound encoder) instead of the dense VL provider.

  • A :meth:mapping_registry override that adds the temporal video_embedder parameter and the sound projection / sound encoder parameters (the latter via a single ** wildcard, since the Megatron sound encoder is HF transformers’ ParakeetEncoder and the parameter names line up 1:1 with sound_encoder.encoder.*).

  • ADDITIONAL_FILE_PATTERNS covering the bespoke Omni HF modeling / processing / audio files that need to be copied during HF export.

Module Contents#

Classes#

NemotronOmniBridge

Bridge for the canonical expanded-sequence Nemotron-3 Omni model.

NemotronOmniLlavaBridge

Deprecated fallback bridge for the historical collapse/expand model.

Functions#

_copy_mapping_with_prefixes

Copy a mapping while preserving its conversion implementation.

API#

bridge.models.nemotron_omni.nemotron_omni_bridge._copy_mapping_with_prefixes(
mapping,
*,
megatron_prefix: str,
hf_prefix: str,
)#

Copy a mapping while preserving its conversion implementation.

class bridge.models.nemotron_omni.nemotron_omni_bridge.NemotronOmniBridge#

Bases: megatron.bridge.models.nemotron_vl.nemotron_vl_bridge.NemotronVLBridge

Bridge for the canonical expanded-sequence Nemotron-3 Omni model.

_HF_PASSTHROUGH_KEYS#

(‘sound_encoder.encoder.feature_extractor.featurizer.fb’, ‘sound_encoder.encoder.feature_extractor.f…

CONFIG_MAPPING#

None

ADDITIONAL_FILE_PATTERNS#

[‘modeling*.py’, ‘configuration*.py’, ‘processing*.py’, ‘processing_utils.py’, ‘image_processing*.py…

provider_bridge(
hf_pretrained: megatron.bridge.models.hf_pretrained.causal_lm.PreTrainedCausalLM,
) → megatron.bridge.models.nemotron_omni.nemotron_omni_provider.NemotronOmniModelProvider#

Create a NemotronOmniModelProvider from the HF Omni config.

Always returns an Omni provider (MoE language model + RADIO ViT vision + optional Parakeet sound encoder). The presence of sound_config is the Hugging Face checkpoint’s sound capability.

classmethod megatron_to_hf_config(provider) → dict#

Export sound capability consistently with model construction.

_llava_mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry#

Build mappings for the historical LLaVA wrapper namespace.

mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry#

Return top-level media mappings plus prefixed NemotronH mappings.

stream_weights_megatron_to_hf(
megatron_model: megatron.bridge.models.nemotron_omni.modeling_nemotron_omni.NemotronOmniModel | list[megatron.bridge.models.nemotron_omni.modeling_nemotron_omni.NemotronOmniModel],
hf_pretrained: megatron.bridge.models.hf_pretrained.causal_lm.PreTrainedCausalLM,
cpu: bool = True,
show_progress: bool = True,
conversion_tasks: list[megatron.bridge.models.conversion.model_bridge.WeightConversionTask] | None = None,
merge_adapter_weights: bool = True,
weight_dtype: torch.dtype | None = None,
) → collections.abc.Iterable[megatron.bridge.models.conversion.model_bridge.HFWeightTuple]#

Export model weights and preserve immutable source-only buffers.

class bridge.models.nemotron_omni.nemotron_omni_bridge.NemotronOmniLlavaBridge#

Bases: bridge.models.nemotron_omni.nemotron_omni_bridge.NemotronOmniBridge

Deprecated fallback bridge for the historical collapse/expand model.

Use :class:NemotronOmniBridge, which is the canonical AutoBridge registration and consumes processor-expanded media-token sequences.

provider_bridge(
hf_pretrained: megatron.bridge.models.hf_pretrained.causal_lm.PreTrainedCausalLM,
) → megatron.bridge.models.nemotron_omni.nemotron_omni_provider.NemotronOmniLlavaModelProvider#
mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry#