> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server. # Nemotron 3.5 Super VL > Nemotron 3.5 Super VL model coverage: architecture, Hugging Face checkpoints, and NeMo AutoModel fine-tuning recipes. [NVIDIA-Nemotron-3.5-Super-midtrain-67B-vision-pretrained](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-midtrain-67B-vision-pretrained) pairs a Nemotron Super NemotronH (hybrid Mamba-2 and Attention) Mixture-of-Experts (MoE) language backbone with a RADIO v2.5-H vision encoder, sharing its `NemotronH_Omni_Reasoning_V3` architecture and vision stack with [Nemotron-3-Nano-Omni](/model-coverage/omni/nvidia/nemotron-omni). This midtrain checkpoint is vision-pretrained only (`sound_config` is absent). There is no audio encoder, so audio inputs are not supported on this checkpoint. | | | | ---------------- | --------------------------------------------------------------------------------- | | **Task** | Multimodal (Text·Image) | | **Architecture** | `NemotronH_Omni_Reasoning_V3` | | **Parameters** | 121B total (MoE, 512 routed experts, 22 active per token; 124B with the MTP head) | | **HF Org** | [nvidia](https://huggingface.co/nvidia) | ## Available Models * **NVIDIA-Nemotron-3.5-Super-midtrain-67B-vision-pretrained**: 121B total (MoE), vision-pretrained, no audio encoder ## Architecture * `NemotronH_Omni_Reasoning_V3` reuses the same `NemotronOmniForConditionalGeneration` wrapper, vision tower, and state-dict adapter as `NemotronH_Nano_Omni_Reasoning_V3`; only the LLM sub-config scale differs (Super compared to Nano). The `sound_encoder` and `sound_projection` submodules are constructed only when the checkpoint's `sound_config` is present, so this checkpoint loads with audio disabled. ## Example Recipes | Recipe | Dataset | Description | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------- | | [nemotron\_3\_5\_super\_vl\_tulu3\_text\_100steps.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_tulu3_text_100steps.yaml) | Tulu-3 | Full SFT, text-only (vision/audio towers frozen and unused) | | [nemotron\_3\_5\_super\_vl\_tulu3\_packed\_cp8\_100steps.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_tulu3_packed_cp8_100steps.yaml) | Tulu-3 (packed, THD) | Context-parallel parity test, `cp_size=8`; text-only packs (`inject_fake_images: false`) | | [nemotron\_3\_5\_super\_vl\_cord\_v2\_peft.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_cord_v2_peft.yaml) | CORD-v2 | LoRA (rank 64) on the LLM linear projections, base model frozen; receipt image to structured field tokens; single node | ### LoRA on CORD-v2 `nemotron_3_5_super_vl_cord_v2_peft.yaml` fine-tunes the model on CORD-v2 receipt field extraction (the same task as the [NemotronOmni CORD-v2 guide](/recipes-e2e-examples/nemotron-omni)) with LoRA adapters instead of full SFT. Because the base weights are frozen, there are no fp32 master weights or optimizer states for them, so the 121B model trains on a single node of 8 H100 (`cp_size 1`, `ep_size 8`) instead of the 4 nodes that full SFT needs. | | | | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | LoRA targets | 272 linear projections: Mamba `in_proj`/`out_proj`, attention `q/k/v/o_proj`, MoE `fc1/fc2_latent_proj` and shared experts (routed experts, vision tower, projector, `lm_head` frozen) | | Trainable parameters | 177M (0.15%), rank 64, alpha 128; TE `FusedAdam` with fp32 master weights, lr 2e-4 | | Peak memory | 38.8 GiB/GPU | | Adapter checkpoint | 355 MB (`adapter_model.safetensors` + `adapter_config.json`) | | Training | 400 steps (global batch 8, 4 epochs of the 800 training receipts) in 13 min on one node; train loss 0.51 → 0.01 | | Validation loss | 0.041 / 0.039 / 0.043 / 0.040 at steps 99 / 199 / 299 / 399 | | Field extraction (first 20 validation receipts, greedy, adapter merged into the base model) | step 199 (`LOWEST_VAL`): 8/20 exact match, mean similarity 0.954 (median 0.997), 20/20 structured outputs | ## Try with NeMo AutoModel **1. Clone and install from source** ([full instructions](/get-started/installation)): ```bash git clone https://github.com/NVIDIA-NeMo/Automodel.git cd Automodel uv sync --locked --all-groups --all-extras --extra vlm-media ``` **2. Run a recipe** from inside the repo — LoRA on CORD-v2 on a single node (adapter-only checkpoints, \~39 GiB/GPU): ```bash uv run automodel examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_cord_v2_peft.yaml --nproc-per-node 8 ``` The Tulu-3 text recipe runs on 4 nodes: ```bash uv run automodel examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_tulu3_text_100steps.yaml --nnodes 4 --nproc-per-node 8 ```