Nemotron-3 Nano Omni#
NVIDIA Nemotron-3 Nano Omni is a 30B-A3B MoE multimodal reasoning model that jointly processes image, video, audio, and text inputs. It pairs a MoE Mamba/attention hybrid language backbone with a RADIO vision tower (static-resolution image path or dynamic-resolution temporal video embedder) and a Parakeet sound encoder.
NeMo Megatron Bridge supports HF↔Megatron conversion, full SFT, and LoRA PEFT on image-text and audio-video-text datasets. The finetuned model can be re-exported to 🤗 Hugging Face format for downstream evaluation or deployment.
Important
Day-0 release. This model is supported on dedicated public branches:
Repo |
Branch |
|---|---|
Megatron-Bridge |
|
Megatron-LM (submodule at |
For the full setup, conversion, inference, training, evaluation, and LoRA
merge / adapter export workflows, see
examples/models/nemotron/nemotron_3_nano_omni/README.md.
Verified configurations#
Choose an exact recorded configuration to see its command and expected result. These selectors are generated from the authoritative verification cards and never synthesize combinations.
Run a configuration#
Choose a workflow, precision, and exact recorded combination. The command and expected result update below.
Import · CPU
✓ VerifiedExact command
./scripts/conversion/convert.sh import --executor slurm --device cpu --nodes 1 --hf-model nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 --hf-revision 24e67ea000b7c2837fc8f9488aa2008524fac8ba --megatron-path work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/cpu-megatron-clean --torch-dtype bfloat16 --tp 1 --pp 1 --ep 1 --etp 1 --trust-remote-codeExpected result
The offline, immutable-revision CPU import exits successfully after mapping 7,333 model parameters and creates a reloadable iter_0000000 torch_dist checkpoint with 33,015,546,816 parameters on the single model parallel rank.
Import · GPU
✓ VerifiedExact command
./scripts/conversion/convert.sh import --executor slurm --device gpu --nodes 1 --gpus-per-node 8 --hf-model nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 --hf-revision 24e67ea000b7c2837fc8f9488aa2008524fac8ba --megatron-path work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/gpu-megatron-clean --torch-dtype bfloat16 --tp 2 --pp 1 --ep 4 --etp 1 --trust-remote-code --low-memory-saveExpected result
The command exits successfully at TP2/PP1/EP4/ETP1 and creates a reloadable iter_0000000 checkpoint. The paired strict GPU export contains the source checkpoint's exact 7,349-key set, shapes, dtypes, and values.
Export · CPU
✓ VerifiedExact command
./scripts/conversion/convert.sh export --executor slurm --device cpu --nodes 1 --hf-model nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 --hf-revision 24e67ea000b7c2837fc8f9488aa2008524fac8ba --megatron-path work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/cpu-megatron-clean/iter_0000000 --hf-path work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/cpu-hf-export-clean --torch-dtype bfloat16 --trust-remote-codeExpected result
CPU export completes in 14 indexed shards totaling 66,031,270,520 bytes. Its exact comparison contains all 7,349 source tensors and 33,015,632,238 values, with identical keys, shapes, dtypes, and values and no dtype conversions. Transformers reloads the local custom-code export natively with the source-equivalent warning set: one generated vision summary buffer is missing and four source-only audio/vision buffers are unexpected.
Export · GPU
✓ VerifiedExact command
./scripts/conversion/convert.sh export --executor slurm --device gpu --nodes 1 --gpus-per-node 8 --hf-model nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 --hf-revision 24e67ea000b7c2837fc8f9488aa2008524fac8ba --megatron-path work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/gpu-megatron-clean/iter_0000000 --hf-path work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/gpu-hf-export-clean --torch-dtype bfloat16 --tp 2 --pp 1 --ep 4 --etp 1 --trust-remote-code --distributed-save --save-every-n-ranks 1Expected result
Distributed export completes in 17 indexed shards totaling 66,031,270,520 bytes. All 7,349 tensors and 33,015,632,238 values match the immutable HF source exactly in keys, shapes, dtypes, and values with no dtype conversions. Transformers reloads the local custom-code export natively with the same one missing generated buffer and four unexpected source-only buffers reported when reloading the pinned source itself.
Pretrain
× UnsupportedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
Megatron Bridge does not publish a pretraining recipe for this multimodal conditional-generation model; the supported package provides supervised CORD v2 SFT and LoRA PEFT workflows.
SFT · H100
✓ VerifiedRecorded metrics
- Initial loss
- 1.281308
- Final loss
- 0.03330838
- Step time · last 10 avg
- 3,603.670 ms
- Model throughput · last 10 avg
- 197.100 TFLOP/s/GPU
- Token throughput · last 10 avg
- 9,092.952 tokens/s/GPU
Exact command
./scripts/training/train.sh --wait --nodes 1 --gpus-per-node 8 --recipe nemotron_omni_cord_v2_sft_8gpu_h100_bf16_config --mode sft --step_func nemotron_omni_step --pretrained_checkpoint work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/gpu-megatron-clean/iter_0000000 --max_steps 100 --save_dir work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/sft-4k-checkpoints --save_interval 50 'dataset.source.load_kwargs={revision:"7f0115a4b758a71d6473b8d085751692da2fef98"}' dataset.do_validation=false dataset.do_test=false validation.eval_iters=0 validation.eval_interval=0 checkpoint.load=null logger.log_interval=1 logger.log_throughput=true rng.seed=5678Expected result
The immutable-revision, real image-text CORD v2 run completes exactly 100 full-SFT optimizer steps on eight H100 GPUs at TP2/PP1/CP1/EP8/ETP1 and GBS/MBS 64/4 with natural-routing HybridEP. Execution uses selective MoE/layernorm recompute, fused attention, TE cross entropy, fused weighted squared-ReLU, and router fusion without changing the image-text objective or routing policy. Model parameters, main gradients, and Adam moments use BF16; optimizer main parameters use scaled FP16 without parameter remainders. LM loss is finite from 1.281308 to 0.03330838, the final ten steps average 3,603.67 ms and 197.10 TFLOP/s/GPU, sampled peak memory is 76,349 MiB, and no iteration is skipped or NaN. Complete optimizer and RNG checkpoints are saved at steps 50 and 100. Direct step-50 resume loads the complete state and reaches step 100 at 197.07 final-ten TFLOP/s/GPU with finite loss and zero skipped or NaN iterations.
Long Context · H100
✓ VerifiedRecorded metrics
- Initial loss
- 1.251364
- Final loss
- 0.01020405
- Step time · last 10 avg
- 15,577.900 ms
- Model throughput · last 10 avg
- 93.730 TFLOP/s/GPU
- Token throughput · last 10 avg
- 4,206.986 tokens/s/GPU
Exact command
./scripts/training/train.sh --wait --nodes 1 --gpus-per-node 8 --recipe nemotron_omni_cord_v2_long_context_sft_8gpu_h100_bf16_config --mode sft --step_func nemotron_omni_step --pretrained_checkpoint work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/gpu-megatron-clean/iter_0000000 --max_steps 100 --save_dir work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/sft-8k-packed-checkpoints --save_interval 50 'dataset.source.load_kwargs={revision:"7f0115a4b758a71d6473b8d085751692da2fef98"}' dataset.do_validation=false dataset.do_test=false validation.eval_iters=0 validation.eval_interval=0 checkpoint.load=null logger.log_interval=1 logger.log_throughput=true rng.seed=5678Expected result
The immutable-revision, real image-text CORD v2 run completes exactly 100 full-SFT optimizer steps on eight H100 GPUs at TP4/PP1/CP2/EP8/ETP1 and GBS/MBS 64/2 with 8K in-batch packing and the standard all-to-all dispatcher. Model parameters, main gradients, and Adam moments use BF16; optimizer main parameters use scaled FP16 without parameter remainders. LM loss is finite from 1.251364 to 0.01020405, the final ten steps average 15,577.9 ms and 93.73 TFLOP/s/GPU, sampled peak memory is 75,197 MiB, and no iteration is skipped or NaN. Complete optimizer and RNG checkpoints are saved at steps 50 and 100. Direct step-50 resume loads the complete state and reaches step 100 with finite loss and zero skipped or NaN iterations.
LoRA · H100
✓ VerifiedRecorded metrics
- Initial loss
- 1.282941
- Final loss
- 0.02634745
- Step time · last 10 avg
- 11,621.420 ms
- Model throughput · last 10 avg
- 61.250 TFLOP/s/GPU
- Token throughput · last 10 avg
- 2,819.621 tokens/s/GPU
Exact command
./scripts/training/train.sh --wait --nodes 1 --gpus-per-node 8 --recipe nemotron_omni_cord_v2_peft_8gpu_h100_bf16_config --mode lora --step_func nemotron_omni_step --pretrained_checkpoint work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/gpu-megatron-clean/iter_0000000 --max_steps 100 --save_dir work/model-verification/nemotron-3-nano-omni-30b-a3b-reasoning/peft-checkpoints --save_interval 50 'dataset.source.load_kwargs={revision:"7f0115a4b758a71d6473b8d085751692da2fef98"}' dataset.do_validation=false dataset.do_test=false validation.eval_iters=0 validation.eval_interval=0 checkpoint.load=null logger.log_interval=1 logger.log_throughput=true rng.seed=5678Expected result
The immutable-revision, real image-text CORD v2 run completes exactly 100 LoRA optimizer steps on eight H100 GPUs at TP2/PP1/CP1/EP8/ETP1 and GBS/MBS 64/1 with natural-routing HybridEP. Model parameters, main gradients, and Adam moments use BF16; optimizer main parameters use scaled FP16 without parameter remainders. LM loss is finite from 1.282941 to 0.02634745, the final ten steps average 11,621.42 ms and 61.25 TFLOP/s/GPU, sampled peak memory is 44,873 MiB, and no iteration is skipped or NaN. Complete optimizer and RNG checkpoints are saved at steps 50 and 100. Direct step-50 resume loads the complete state and reaches step 100 with finite loss and zero skipped or NaN iterations.