Nemotron 3 Nano Omni#
This guide explains how to post-train the Nemotron 3 Nano Omni vision-language model with GRPO using NeMo RL. Both the AutoModel and Megatron backends are supported for image-and-text training.
Multimodal payload deduplication#
The maintained Nemotron recipes enable grpo.deduplicate_multimodal_data to
share immutable model-ready media segments across logical GRPO generations and
re-intern them after batching, replay, and sharding. The representation supports
image, video, and audio payload keys, although the maintained recipes currently
qualify image inputs only. Deduplication currently requires the vLLM generation
backend and data_plane.enabled=false.
grpo.debug_payload_metrics emits logical, physical, and protocol-5 serialized
payload sizes for the exact Ray boundaries used by generation, replay, logprobs,
and training. It is disabled by default because measuring serialized size adds
work and is intended for qualification and debugging rather than production.
AutoModel backend#
It covers two recipes:
CLEVR-CoGenT — runs on a single 8-GPU node (interactive container).
MMPR-Tiny — runs on 4 nodes via Slurm.
Both share the same checkpoint, model code, and reward pipeline; they differ only in the dataset, reward functions, and node count.
Recipe 1 — CLEVR-CoGenT (single-node)#
The CLEVR-CoGenT recipe uses examples/configs/recipes/vlm/vlm_grpo-nemotron-omni-30ba3b-clevr-1n8g-automodel-ep8.v1.yaml. It expects 8 GPUs on a single node, EP=8 across the experts, and TP=8 in vLLM.
Key knobs in the config:
Field |
Value |
|---|---|
|
path to the Nemotron-Omni HF checkpoint |
|
8 |
|
8 |
|
8192 |
|
|
|
|
|
|
CLEVR is loaded automatically from HuggingFace by the clevr-cogent response dataset on first run; no manual prep is required.
Launch (interactive container)#
From inside the container on an 8-GPU node:
export NRL_MAMBA_PREFILL_DECODE_SYNC="${NRL_MAMBA_PREFILL_DECODE_SYNC:-1}"
uv run examples/run_vlm_grpo.py --config examples/configs/recipes/vlm/vlm_grpo-nemotron-omni-30ba3b-clevr-1n8g-automodel-ep8.v1.yaml \
cluster.gpus_per_node=8 \
cluster.num_nodes=1
To override the model path or any other YAML field, append Hydra-style overrides:
uv run examples/run_vlm_grpo.py --config examples/configs/recipes/vlm/vlm_grpo-nemotron-omni-30ba3b-clevr-1n8g-automodel-ep8.v1.yaml \
policy.model_name=/path/to/your/checkpoint \
cluster.gpus_per_node=8 cluster.num_nodes=1
Recipe 2 — MMPR-Tiny (4-node Slurm)#
The MMPR-Tiny recipe uses examples/configs/recipes/vlm/vlm_grpo-nemotron-omni-30ba3b-mmpr-4n8g-automodel-ep8.v1.yaml. Differences vs. the CLEVR recipe:
Field |
Value |
|---|---|
|
|
|
local cache dir for MMPR-Tiny (loader auto-downloads from HF) |
|
|
|
8192 |
|
|
Launch (4-node Slurm)#
Submit with ray.sub. From the repo root on a Slurm login node:
# --- Cluster config ---
export SBATCH_ACCOUNT=your_slurm_account
export SBATCH_PARTITION=batch
export SBATCH_TIME=4:00:00
export CONTAINER=/path/to/containers/nemo-rl-nano-v3-vl-<tag>.sqsh
export MOUNTS=/lustre:/lustre
export HF_HOME=/path/to/cache/huggingface
export TMPDIR=/tmp/nrl-${USER}
export NCCL_DEBUG=WARN
export NRL_IGNORE_VERSION_MISMATCH=1
# --- Run config ---
NUM_NODES=4
GPUS_PER_NODE=8
JOB_NAME=grpo-nemotron-omni-mmpr
RESULTS_DIR=$PWD/results/${JOB_NAME}
CONFIG_PATH=examples/configs/recipes/vlm/vlm_grpo-nemotron-omni-30ba3b-mmpr-4n8g-automodel-ep8.v1.yaml
# --- Build the training command (run inside the container on every node) ---
export COMMAND="\
export PYTHONPATH=\${PYTHONPATH:-}:/path/to/automodel-omni && \
export CUDA_LAUNCH_BLOCKING=0 && \
export TORCH_USE_CUDA_DSA=0 && \
export NRL_MAMBA_PREFILL_DECODE_SYNC=1 && \
mkdir -p ${HF_HOME} ${TMPDIR} ${RESULTS_DIR} && \
uv run examples/run_vlm_grpo.py --config ${CONFIG_PATH} \
cluster.num_nodes=${NUM_NODES} \
cluster.gpus_per_node=${GPUS_PER_NODE} \
checkpointing.checkpoint_dir='${RESULTS_DIR}' \
logger.wandb.name='${JOB_NAME}'"
# --- Submit ---
sbatch \
--nodes=${NUM_NODES} \
--account=${SBATCH_ACCOUNT} \
--job-name=nemo-rl-${JOB_NAME} \
--partition=${SBATCH_PARTITION} \
--time=${SBATCH_TIME} \
--dependency=singleton \
--gres=gpu:${GPUS_PER_NODE} \
ray.sub
To run on a different node count, change NUM_NODES and the --nodes flag.
Megatron backend#
The Megatron backend uses a dedicated NemotronOmniModel supplied by Megatron Bridge. The Hugging Face processor expands each image placeholder into the complete media-token sequence before the batch reaches the model. NeMo RL passes that expanded sequence and the image tensors to the model; NemotronOmniModel replaces the media-token positions with RADIO encoder outputs and then performs sequence packing and context-parallel sharding.
This is the same model-owned packing boundary used by maintained Megatron VLM integrations. It differs from the historical Nemotron Omni LLaVAModel path, which collapsed the expanded media-token sequence before packing and expanded it again inside the model. The dedicated model removes that extra representation change and allows the integration to use Megatron Bridge and Megatron-LM from their maintained main branches.
The maintained Megatron VLM recipes cover Nano image-and-text GRPO. The NeMo Gym integration also supports the static, one-video-per-row workflow described below. Audio and mixed audio-video rows are not supported by that workflow.
Checkpoint compatibility#
Use the nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 Hugging Face checkpoint or a checkpoint converted with the dedicated NemotronOmniModel integration. Legacy Megatron checkpoints whose parameter names use an llava_model prefix are not compatible with this model definition. Reconvert those checkpoints from the original Hugging Face checkpoint instead of loading them directly.
Maintained recipes#
Workload |
Recipe |
Topology |
|---|---|---|
CLEVR-CoGenT |
|
1 node, 8 GPUs, TP=8, EP=8 |
MMPR-Tiny |
|
4 nodes, 8 GPUs per node, TP=8, EP=16, vLLM TP=2 |
Launch the single-node Megatron recipe from inside the container on an 8-GPU node:
uv run examples/run_vlm_grpo.py \
--config examples/configs/recipes/vlm/vlm_grpo-nemotron-omni-30ba3b-clevr-1n8g-megatron-tp8ep8.v1.yaml
For a four-node Slurm run, use the ray.sub example above with the following configuration path and omit the AutoModel-specific PYTHONPATH addition:
CONFIG_PATH=examples/configs/recipes/vlm/vlm_grpo-nemotron-omni-30ba3b-mmpr-4n8g-megatron-tp8ep16.v1.yaml
The recipes keep sequence packing enabled because the model owns the packing step after multimodal embedding insertion. They also request raw generation log probabilities so that vLLM and the Megatron policy compare the same pre-processor probability values when generation constraints such as bad_words are active. The generation context cap prevents the processor-expanded image prompt plus generated response from exceeding the configured 8192-token context length.
NeMo Gym video GRPO#
Nemotron 3 Nano Omni video GRPO accepts one local video in the initial static
NeMo Gym prompt. The user message uses an input_video part followed by
input_text:
{
"responses_create_params": {
"input": [{
"role": "user",
"content": [
{"type": "input_video", "video_url": "/absolute/path/clip.mp4"},
{"type": "input_text", "text": "Which option describes the clip?"}
]
}]
}
}
Only local paths under policy.generation.vllm_kwargs.allowed_local_media_path
are accepted. Environment-produced video and video introduced in a later
trajectory turn are not supported. Each row must contain exactly one video and no audio. MCQA rows
must provide real top-level options and one uppercase expected_answer
present in those options. Use
prepare_video_dataset.py
to convert and validate source JSONL. Raw videos are decoded with TorchCodec;
the runtime can also read externally prepared lossless frame manifests, but the
converter currently emits raw-video rows only.
The recipes contain explicit placeholder paths. Override them when launching:
uv run examples/nemo_gym/run_grpo_nemo_gym.py \
--config examples/configs/recipes/vlm/vlm_grpo-nemotron-omni-30ba3b-2n8g-megatron-tp4ep4-gym-video.v1.yaml \
policy.generation.vllm_kwargs.allowed_local_media_path=/path/to/video-data \
data.train.data_path=/path/to/train.jsonl \
data.validation.data_path=/path/to/validation.jsonl
Policy and rollout preprocessing must use the same sampling contract. The
provided recipes set one policy.generation.vllm_cfg.video block for
TorchCodec-backed Nemotron sampling, 32 frames, and a temporal patch size of 2.
NeMo RL materializes those values for both policy and rollout preprocessing.
The runtime therefore needs TorchCodec and its FFmpeg dependencies; no parallel
sampling environment variables are required.
The remaining policy-side video settings live under data.default:
video_target_num_patches, video_maintain_aspect_ratio, and, for generic
processors only, min_generation_tokens. Set
policy.generation.vllm_cfg.reset_encoder_cache_after_weight_update: true only
when the multimodal encoder trains; the provided frozen-vision recipes leave it
disabled.
The synchronous and asynchronous overlays are:
They require the corresponding Nemotron Omni support in Megatron Bridge and video request/token propagation in NeMo Gym. Policy preprocessing numerically matches unmodified stock vLLM 0.25.1; a custom vLLM fork is not required.
Both overlays use a large positive grpo.max_num_steps value so it does not
bind normal training. Training still follows the existing GRPO step-limit
semantics. The recipes also leave
grpo.seq_logprob_error_threshold: null: logged token multiplicative
probability error (TMPE) is raw and no high-error sequence is masked. Interpret
TMPE together with reward, loss, sequence length, and refit metrics; isolated
maxima are less informative than a sustained shift in the distribution.