> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/curator/llms-full.txt.

# nemo_curator.models.nemotron_h_vl

## Module Contents

### Classes

| Name                                                                  | Description                                                |
| --------------------------------------------------------------------- | ---------------------------------------------------------- |
| [`LLM`](#nemo_curator-models-nemotron_h_vl-LLM)                       | -                                                          |
| [`NemotronHVL`](#nemo_curator-models-nemotron_h_vl-NemotronHVL)       | NemotronH hybrid Mamba-Attention VLM for video captioning. |
| [`SamplingParams`](#nemo_curator-models-nemotron_h_vl-SamplingParams) | -                                                          |

### Data

[`EXPECTED_VIDEO_TAG_PARTS`](#nemo_curator-models-nemotron_h_vl-EXPECTED_VIDEO_TAG_PARTS)

[`NemotronVariant`](#nemo_curator-models-nemotron_h_vl-NemotronVariant)

[`VIDEO_TAG_SPLIT_MAX`](#nemo_curator-models-nemotron_h_vl-VIDEO_TAG_SPLIT_MAX)

[`VLLM_AVAILABLE`](#nemo_curator-models-nemotron_h_vl-VLLM_AVAILABLE)

[`_NEMOTRON_REVISION_INFO`](#nemo_curator-models-nemotron_h_vl-_NEMOTRON_REVISION_INFO)

[`_NEMOTRON_VARIANTS_INFO`](#nemo_curator-models-nemotron_h_vl-_NEMOTRON_VARIANTS_INFO)

### API

<Anchor id="nemo_curator-models-nemotron_h_vl-LLM">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_curator.models.nemotron_h_vl.LLM()
    ```
  </CodeBlock>
</Anchor>

<Indent />

<Anchor id="nemo_curator-models-nemotron_h_vl-NemotronHVL">
  <CodeBlock links={{"nemo_curator.models.nemotron_h_vl.NemotronVariant":"#nemo_curator-models-nemotron_h_vl-NemotronVariant"}} showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_curator.models.nemotron_h_vl.NemotronHVL(
        model_dir: str,
        model_variant: nemo_curator.models.nemotron_h_vl.NemotronVariant = 'nemotron',
        caption_batch_size: int = 8,
        max_output_tokens: int = 512,
        stage2_prompt_text: str | None = None,
        verbose: bool = False
    )
    ```
  </CodeBlock>
</Anchor>

<Indent>
  **Bases:** [ModelInterface](/nemo-curator/nemo_curator/models/base#nemo_curator-models-base-ModelInterface)

  NemotronH hybrid Mamba-Attention VLM for video captioning.

  Supports multiple checkpoint variants from HuggingFace:

  * nemotron / nemotron-bf16: BF16 precision (default)
  * nemotron-fp8: FP8 quantized
  * nemotron-nvfp4: NVFP4 quantized

  Models are automatically downloaded from HuggingFace on first use.

  <ParamField path="_hf_model_id" type="= _NEMOTRON_VARIANTS_INFO[self._normalized_variant]" />

  <ParamField path="_normalized_variant" type="NemotronVariant = 'nemotron-bf16'" />

  <ParamField path="model_id_names" type="list[str]">
    Return HuggingFace model ID for the selected variant.
  </ParamField>

  <ParamField path="stage2_prompt" />

  <ParamField path="weight_file" type="= str(Path(model_dir) / self._hf_model_id)" />

  <Anchor id="nemo_curator-models-nemotron_h_vl-NemotronHVL-_refine_caption_prompt">
    <CodeBlock showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_curator.models.nemotron_h_vl.NemotronHVL._refine_caption_prompt(
          original_prompt: str,
          refinement_text: str
      ) -> str
      ```
    </CodeBlock>
  </Anchor>

  <Indent>
    Create a refined prompt for stage 2 captioning.
  </Indent>

  <Anchor id="nemo_curator-models-nemotron_h_vl-NemotronHVL-download_weights_on_node">
    <CodeBlock links={{"nemo_curator.models.nemotron_h_vl.NemotronVariant":"#nemo_curator-models-nemotron_h_vl-NemotronVariant"}} showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_curator.models.nemotron_h_vl.NemotronHVL.download_weights_on_node(
          model_dir: str,
          variant: nemo_curator.models.nemotron_h_vl.NemotronVariant = 'nemotron'
      ) -> None
      ```
    </CodeBlock>
  </Anchor>

  <Indent>
    <Badge>
      classmethod
    </Badge>

    Download NemotronH VL weights from HuggingFace.

    Models are automatically downloaded from HuggingFace Hub on first use.
    Supports multiple quantization variants for different performance/memory tradeoffs.

    **Parameters:**

    <ParamField path="model_dir" type="str">
      Base directory for model weights. The model will be downloaded
      to a subdirectory named after the HuggingFace model ID.
    </ParamField>

    <ParamField path="variant" type="NemotronVariant" default="'nemotron'">
      Model variant to download. Options:

      * "nemotron" or "nemotron-bf16": BF16 precision (default)
      * "nemotron-fp8": FP8 quantized
      * "nemotron-nvfp4": NVFP4 quantized
    </ParamField>
  </Indent>

  <Anchor id="nemo_curator-models-nemotron_h_vl-NemotronHVL-generate">
    <CodeBlock showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_curator.models.nemotron_h_vl.NemotronHVL.generate(
          videos: list[dict[str, typing.Any]],
          generate_stage2_caption: bool = False,
          batch_size: int = 16
      ) -> list[str]
      ```
    </CodeBlock>
  </Anchor>

  <Indent />

  <Anchor id="nemo_curator-models-nemotron_h_vl-NemotronHVL-setup">
    <CodeBlock showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_curator.models.nemotron_h_vl.NemotronHVL.setup() -> None
      ```
    </CodeBlock>
  </Anchor>

  <Indent />
</Indent>

<Anchor id="nemo_curator-models-nemotron_h_vl-SamplingParams">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_curator.models.nemotron_h_vl.SamplingParams()
    ```
  </CodeBlock>
</Anchor>

<Indent />

<Anchor id="nemo_curator-models-nemotron_h_vl-EXPECTED_VIDEO_TAG_PARTS">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.models.nemotron_h_vl.EXPECTED_VIDEO_TAG_PARTS = 2
    ```
  </CodeBlock>
</Anchor>

<Anchor id="nemo_curator-models-nemotron_h_vl-NemotronVariant">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.models.nemotron_h_vl.NemotronVariant = Literal['nemotron', 'nemotron-bf16', 'nemotron-fp8', 'nemotron-nvfp4']
    ```
  </CodeBlock>
</Anchor>

<Anchor id="nemo_curator-models-nemotron_h_vl-VIDEO_TAG_SPLIT_MAX">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.models.nemotron_h_vl.VIDEO_TAG_SPLIT_MAX = 1
    ```
  </CodeBlock>
</Anchor>

<Anchor id="nemo_curator-models-nemotron_h_vl-VLLM_AVAILABLE">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.models.nemotron_h_vl.VLLM_AVAILABLE = True
    ```
  </CodeBlock>
</Anchor>

<Anchor id="nemo_curator-models-nemotron_h_vl-_NEMOTRON_REVISION_INFO">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.models.nemotron_h_vl._NEMOTRON_REVISION_INFO: Final = {'nemotron': '5d250e2e111dc5e1434131bdf3d590c27a878ade', 'nemotron-bf16': '5d250...
    ```
  </CodeBlock>
</Anchor>

<Anchor id="nemo_curator-models-nemotron_h_vl-_NEMOTRON_VARIANTS_INFO">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.models.nemotron_h_vl._NEMOTRON_VARIANTS_INFO: Final = {'nemotron': 'nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16', 'nemotron-bf16': 'nvi...
    ```
  </CodeBlock>
</Anchor>