bridge.models.nemotron_omni.inference_inputs

Native HF preprocessing adapted to the Omni family’s packed vision input.

This is an inference adapter, not the fixed-canvas training preprocessing policy. Both backends consume one processor invocation’s pixels and expanded token IDs.

Module Contents

Classes

NemotronOmniInputs

Native HF inputs and their equivalent packed Bridge media tensors.

Functions

is_nemotron_omni	Recognize Omni and Super-VL by registered model type or architecture.
load_nemotron_omni_video	Decode frames with the maintained Omni decoder and retain source timestamps.
prepare_nemotron_omni_inputs	Render native expanded media tokens and pack exactly the processed pixels.
nemotron_omni_reference_metadata	Bind a saved HF reference to its source and exact preprocessed media bytes.

API

class bridge.models.nemotron_omni.inference_inputs.NemotronOmniInputs

Native HF inputs and their equivalent packed Bridge media tensors.

hf: dict[str, Any]

None

bridge: dict[str, torch.Tensor]

None

bridge.models.nemotron_omni.inference_inputs.is_nemotron_omni(config: Any) → bool

Recognize Omni and Super-VL by registered model type or architecture.

bridge.models.nemotron_omni.inference_inputs.load_nemotron_omni_video(

path: str,

*,

fps: float,

) → tuple[list[Any], Any]

Decode frames with the maintained Omni decoder and retain source timestamps.

bridge.models.nemotron_omni.inference_inputs.prepare_nemotron_omni_inputs(

processor: Any,

*,

prompt: str,

images: list[Any] | None = None,

video_frames: list[Any] | None = None,

video_metadata: Any = None,

) → bridge.models.nemotron_omni.inference_inputs.NemotronOmniInputs

Render native expanded media tokens and pack exactly the processed pixels.

Images remain independent image-embedder items. Video frames form one temporal item; a single frame is repeated only for Bridge’s temporal routing. Sizes are post-resize, before temporal grouping. No additional resize or normalization is performed, and no training prompt is synthesized.

bridge.models.nemotron_omni.inference_inputs.nemotron_omni_reference_metadata(

inputs: bridge.models.nemotron_omni.inference_inputs.NemotronOmniInputs,

*,

model: str,

revision: str | None,

video_pruning_rate: float | None = None,

) → dict[str, Any]

Bind a saved HF reference to its source and exact preprocessed media bytes.