Nemotron 3 Ultra#
Technical Report: NVIDIA Nemotron 3 Ultra Technical Report
This guide explains how to post-train the Nemotron 3 Ultra model with NeMo RL on GB200 NVL72 (ARM64 / aarch64) hardware.
Overview#
Nemotron 3 Ultra is post-trained with a multi-stage pipeline that mixes Reinforcement Learning with Verifiable Rewards (RLVR), RLHF, and Multi-Teacher On-Policy Distillation (MOPD) from a panel of specialised teacher models. The main stages are:
Student RLVR — produces the student policy from the supervised fine-tuning (SFT) checkpoint using GRPO with verifiable rewards.
Teacher training — trains a panel of teachers (the Student RLVR policy itself serves as the general teacher, alongside specialised teachers such as reasoning, instruction-following/abstention, RLHF chat, and SWE).
MOPD — on-policy distillation from the student into the teacher panel.
Every stage shares the same launcher (ultra_launch.sh) and a per-stage YAML
config under examples/nemo_gym/nemotron-3-ultra/.
Checkpoint flow#
The pipeline starts from an SFT checkpoint and produces a panel of teacher checkpoints that, together with the Student RLVR output, feed MOPD. The Student RLVR policy itself serves as the general teacher:
┌─────┐
│ SFT │
└──┬──┘
v
┌──────────────┐
│ Student RLVR │
└──┬───────────┘
│
│ ┌────────────────────┐
├──>│ General Teacher │──┐
│ └────────────────────┘ │
│ ┌────────────────────┐ │
├──>│ Reasoning Teacher │──┤
│ └────────────────────┘ │
│ ┌────────────────────┐ │
├──>│ RLHF Teacher │──┤
│ └────────────────────┘ │
│ ┌────────────────────┐ │
├──>│ IFBench Teacher │──┤
│ └────────────────────┘ │
│ ┌────────────────────┐ │
├──>│ SWE Teacher │──┤
│ └────────────────────┘ │
│ │
v v
┌─────────┐ ┌──────────┐
│ Student │──────────────>│ MOPD │
└─────────┘ └──────────┘
Container#
Ultra uses vLLM and requires an aarch64 (arm64) image for GB200 NVL72 nodes. Prebake the NeMo Gym virtual environments used by the recipes to avoid building them when each training job starts. From the root of the repository, build the image with the Gym virtual environments for the Ultra recipes:
docker buildx build \
--platform linux/arm64 \
--progress=plain \
-f docker/Dockerfile \
--target release \
-t <your-registry>/nemo-rl:main-ultra-prefetched-venvs \
--push \
--build-context nemo-rl=. \
--build-arg MAX_JOBS=8 \
--build-arg SKIP_SGLANG_BUILD=1 \
--build-arg SKIP_TRTLLM_BUILD=1 \
--build-arg NEMO_GYM_PREFETCH_CONFIGS="examples/nemo_gym/prefetch_ultra_all_envs.yaml" \
.
Build args:
NEMO_GYM_PREFETCH_CONFIGS— space-separated union configs whose Gym virtual environments are baked into the image.SKIP_SGLANG_BUILD=1— Ultra runs on vLLM; skip the SGLang build.SKIP_TRTLLM_BUILD=1— Ultra does not use TensorRT-LLM; skip its build.MAX_JOBS— parallel build jobs; tune to your machine.--build-context nemo-rl=.— build from your local checkout (otherwise the Dockerfile pullsNVIDIA-NeMo/RL.git#main).
To run on the cluster with Slurm, convert the image to a squashfs (.sqsh)
with enroot:
enroot import -o nemo-rl-container.sqsh \
docker://<your-registry>/nemo-rl:main-ultra-prefetched-venvs
Pass the resulting image as CONTAINER in every launch command below (shown as
CONTAINER=/path/to/nemo-rl-container — a .sqsh path, or a registry image URI
if you’re not using enroot). All Ultra stages run from this single image.
Download and prepare the data#
The training blends are published as
nvidia/Nemotron-RL-Ultra-Training-Blends,
one JSONL per stage: rlvr1, rlvr2, ifbench, rlhf, reasoning, swe, and
mopd. Math rows that originate from BytedTsinghua-SIA/DAPO-Math-17k and
Skywork/Skywork-OR1-RL-Data ship as placeholders; the bundled
fill_placeholders.py restores them from the original datasets on Hugging Face.
export DATA_DIR=/path/to/ultra/data
# 1. Download the blends + fill_placeholders.py
huggingface-cli download nvidia/Nemotron-RL-Ultra-Training-Blends \
--repo-type dataset --local-dir ultra-blends
# 2. Restore the DAPO / Skywork placeholders into $DATA_DIR
cd ultra-blends
./fill_placeholders.py --input-dir . --output-dir "$DATA_DIR" # requires uv
This produces $DATA_DIR/{rlvr1,rlvr2,ifbench,rlhf,reasoning,swe,mopd}.jsonl. Hold
out the last 100 rows of each blend as a validation split — the launch commands
below consume <name>.train.jsonl as TRAIN_PATH and <name>.val.jsonl as
VAL_PATH:
cd "$DATA_DIR"
for name in rlvr1 rlvr2 ifbench rlhf reasoning swe mopd; do
head -n -100 "$name.jsonl" > "$name.train.jsonl"
tail -n 100 "$name.jsonl" > "$name.val.jsonl"
done
By electing to use the external datasets you are responsible for confirming their licenses are fit for your intended use.
The SWE stage additionally requires per-instance .sif container images built
from SWE-Gym and SWE-rebench-V2 — see SWE Teacher for the build
steps.
For now, each stage takes a JSONL training file and a JSONL validation file (see Launch script below).
Prepare the code#
git clone --recursive -b main https://github.com/NVIDIA-NeMo/RL.git
cd RL
Build the sandbox container#
Several Gym environments used during
training (notably ns_tools for stateful Python execution with math
verification, and math_formal_lean for Lean4 proof verification) rely on a
sandbox container. Build it from the
NeMo-Skills Dockerfile:
git clone https://github.com/NVIDIA-NeMo/Skills.git
cd Skills
git checkout b620e79 # Skills commit pinned for the Ultra release
docker build -t nemo-skills-sandbox:latest -f dockerfiles/Dockerfile.sandbox .
For SLURM clusters using enroot, convert
to a .sqsh:
enroot import -o nemo-skills-sandbox.sqsh dockerd://nemo-skills-sandbox:latest
Launch script#
Every stage is submitted with examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh,
run from the repo root. The launcher
handles SLURM submission, code snapshotting, persistent cache management, and
container mounts — stage-specific hyperparameters (batch size, advantage clip,
MoE parallelism, learning rate) live in the per-stage YAML.
Set the following before each bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh invocation:
Variable |
Purpose |
|---|---|
|
Job name, W&B run name, and the suffix for output directories. Must be unique per run; same name across resubmissions resumes from the latest checkpoint. |
|
Path to the per-stage YAML config (e.g. |
|
Initial policy checkpoint (HuggingFace repo id or local path). Student RLVR starts from the Ultra SFT checkpoint; the teacher stages start from the Student RLVR checkpoint; MOPD starts from the student (the Student RLVR checkpoint). |
|
Training data JSONL. |
|
Validation data JSONL. |
|
NeMo RL container image ( |
|
Sandbox image from Build the sandbox container. |
|
Directory on a shared filesystem (e.g. Lustre) where vLLM/Triton/Inductor compile caches are persisted across runs. |
|
Comma-separated |
|
Your SLURM cluster credentials. |
|
GenRM judge: HF repo id or local path. Default is |
|
Optional. URL of a separately-deployed GenRM service (e.g. |
|
NL2Bash / general-purpose judge: HF repo id or local path. Default judge is |
|
Optional. URL of a separately-deployed NL2Bash judge service. Same semantics as |
|
Content-safety judge: HF repo id or local path. Default is |
Serving GenRM outside Gym. For a separately deployed OpenAI-compatible endpoint, set
GENRM_BASE_URL=http://<host>:<port>/v1. Judging is then routed to that endpoint instead of being served from the Gym pool, allowing one deployment to back multiple training runs.To co-schedule dedicated model servers with training in one Slurm heterogeneous allocation, set
EXTERNAL_JUDGES=1— see External judge services below.
Optional knobs:
Variable |
Default |
Purpose |
|---|---|---|
|
|
SLURM |
|
empty |
Optional SLURM flags |
|
|
GB200 4-GPU node split. Total must be a multiple of 16. |
|
|
Set to |
|
from YAML |
Override |
|
unset / |
W&B logging is disabled if |
|
unset |
Shared HuggingFace cache and gated-model token |
|
|
Snapshot the source tree at submission time |
|
|
Set to |
External judge services#
By default the GenRM and NL2Bash judges are served by NeMo Gym inside the
training Ray cluster, consuming part of NUM_GYM_NODES. Setting
EXTERNAL_JUDGES=1 instead serves them as fixed-model vLLM pools in a second
Slurm heterogeneous component. This keeps judge serving off the training Ray
cluster, so a judge restart or OOM cannot destabilise training, and it lets each
judge be scaled independently of the Gym pool.
The launcher registers both pools through the
external Gym vLLM pool helpers
and submits tools/external_gym_vllm/run_in_allocation.sh instead of ray.sub.
That wrapper starts every replica in its own private Ray cluster, brings up one
OpenAI-compatible load balancer per pool, waits for all backends to become
healthy, substitutes the resolved URLs into the driver command, and only then
starts ray.sub on the training component. If a required service exits, the
training job is stopped.
In this mode GENRM_MODEL and NL2BASH_JUDGE_MODEL are the checkpoints the
pools serve, and Gym addresses them by GENRM_SERVED_MODEL_NAME /
NL2BASH_SERVED_MODEL_NAME (both model by default). Setting
GENRM_BASE_URL or NL2BASH_BASE_URL by hand is rejected, since the wrapper
supplies those.
Each pool is registered only when its model variable is set, so set at least
one of them. Stages declare only the judges they use — rlhf_teacher has no
NL2Bash block, reasoning_teacher has no GenRM block, swe_teacher has
neither — and setting a judge the stage does not declare makes the driver fail
on an unknown config key.
BASE_LOG_DIR and EXTERNAL_VLLM_TOOLS_DIR_HOST (the repo’s
tools/external_gym_vllm) must resolve under EXTERNAL_VLLM_SHARED_ROOT,
because the wrapper bind-mounts that root into the service containers. Since
RESULTS_DIR defaults to a path relative to the repo, pass an explicit
BASE_LOG_DIR on the shared filesystem when the checkout lives elsewhere.
Variable |
Default |
Purpose |
|---|---|---|
|
|
|
|
|
Independent DP=1 GenRM servers and TP per server |
|
|
Reasoning parser for the GenRM; |
|
unset |
Path to a reasoning-parser plugin |
|
|
Independent DP=1 NL2Bash servers and TP per server |
|
|
SLURM |
Node counts are derived from the replica shapes, not set directly:
nodes = REPLICAS × TENSOR_PARALLEL_SIZE / GPUS_PER_NODE per pool. The two
components are validated separately — NUM_TRAIN_NODES + NUM_GEN_NODES + NUM_GYM_NODES must be a multiple of SEGMENT_SIZE, and the external node total
must be a multiple of EXTERNAL_VLLM_SEGMENT_SIZE. With both judges external,
NUM_GYM_NODES only has to cover the safety judge. Model paths, the parser
plugin, and BASE_LOG_DIR must live under EXTERNAL_VLLM_SHARED_ROOT
(/lustre by default), which is mounted into the service containers.
INTERACTIVE=1 is not supported in this mode.
Building on the Phase 1 invocation, add:
EXTERNAL_JUDGES=1 \
GENRM_MODEL=/path/to/genrm-checkpoint \
GENRM_REPLICAS=16 \
NL2BASH_JUDGE_MODEL=/path/to/nl2bash-judge-checkpoint \
NL2BASH_REPLICAS=4 \
NUM_TRAIN_NODES=64 \
NUM_GEN_NODES=172 \
NUM_GYM_NODES=4 \
BASE_LOG_DIR=/lustre/path/to/ray_logs \
EXP_NAME=... CONFIG_PATH=... MODEL_PATH=... TRAIN_PATH=... VAL_PATH=... \
CONTAINER=... SANDBOX_CONTAINER=... PERSISTENT_CACHE=... \
SLURM_PARTITION=$SLURM_PARTITION SLURM_ACCOUNT=$SLURM_ACCOUNT \
SAFETY_JUDGE_MODEL=nvidia/Nemotron-Content-Safety-Reasoning-4B \
bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh
That allocates 64 + 172 + 4 = 240 NeMo RL nodes (a multiple of SEGMENT_SIZE,
with Gym now hosting only the safety judge) plus 16 GenRM + 4 NL2Bash = 20
external nodes. Set only GENRM_MODEL or only NL2BASH_JUDGE_MODEL to move
just that judge out of Gym — useful for the teacher stages, which declare only
the judges they use.
Stage 1 — Student RLVR#
GRPO with verifiable rewards on the Ultra SFT checkpoint.
Student RLVR is split into two phases that share the same EXP_NAME so the
second phase resumes from the first phase’s checkpoint:
Phase 1 |
Phase 2 |
|
|---|---|---|
Config |
|
|
|
49,152 |
65,536 |
Steps in this phase |
~128 |
~50 |
|
|
|
|
0.25 |
0.08 |
Both phases share TP=8, EP=64, CP=8, PP=1, GBS=8192 (512 prompts × 16 generations), advantage clip ±20, and the 256-node cluster shape.
Phase 1 — 49k context, 128 steps#
EXP_NAME=ultra-student-rlvr \
CONFIG_PATH=examples/nemo_gym/nemotron-3-ultra/student_rlvr1.yaml \
ENABLE_MTP_INFERENCE=1 \
NRL_MAX_STEPS=128 \
MODEL_PATH=/path/to/ultra_sft_checkpoint \
TRAIN_PATH=$DATA_DIR/rlvr1.train.jsonl \
VAL_PATH=$DATA_DIR/rlvr1.val.jsonl \
CONTAINER=/path/to/nemo-rl-container \
SANDBOX_CONTAINER=/path/to/nemo-skills-sandbox.sqsh \
PERSISTENT_CACHE=/path/to/persistent/cache \
EXTRA_MOUNTS=/lustre:/lustre \
SLURM_PARTITION=$SLURM_PARTITION \
SLURM_ACCOUNT=$SLURM_ACCOUNT \
GENRM_MODEL=nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-GenRM \
NL2BASH_JUDGE_MODEL=Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 \
SAFETY_JUDGE_MODEL=nvidia/Nemotron-Content-Safety-Reasoning-4B \
WANDB_API_KEY=$WANDB_API_KEY \
HF_HOME=/path/to/hf_cache \
HF_TOKEN=$HF_TOKEN \
bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh
Phase 2 — 65k context, ~50 more steps#
Same env vars as Phase 1 but swap the config, raise NRL_MAX_STEPS, and point
at the Phase 2 training data file. Keep EXP_NAME and RESULTS_DIR identical
to Phase 1 so CheckpointManager auto-resumes from the latest Phase 1
checkpoint.
EXP_NAME=ultra-student-rlvr \
CONFIG_PATH=examples/nemo_gym/nemotron-3-ultra/student_rlvr2.yaml \
ENABLE_MTP_INFERENCE=1 \
NRL_MAX_STEPS=178 \
MODEL_PATH=/path/to/ultra_sft_checkpoint \
TRAIN_PATH=$DATA_DIR/rlvr2.train.jsonl \
VAL_PATH=$DATA_DIR/rlvr2.val.jsonl \
CONTAINER=/path/to/nemo-rl-container \
SANDBOX_CONTAINER=/path/to/nemo-skills-sandbox.sqsh \
PERSISTENT_CACHE=/path/to/persistent/cache \
EXTRA_MOUNTS=/lustre:/lustre \
SLURM_PARTITION=$SLURM_PARTITION \
SLURM_ACCOUNT=$SLURM_ACCOUNT \
GENRM_MODEL=nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-GenRM \
NL2BASH_JUDGE_MODEL=Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 \
SAFETY_JUDGE_MODEL=nvidia/Nemotron-Content-Safety-Reasoning-4B \
WANDB_API_KEY=$WANDB_API_KEY \
HF_HOME=/path/to/hf_cache \
HF_TOKEN=$HF_TOKEN \
bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh
Note: Phase 1 and Phase 2 use different training blends (rlvr1.train.jsonl and
rlvr2.train.jsonl); see the Download and prepare the data
section above.
The launcher reports the experiment directory layout, sample monitoring commands, and (on submission) the SLURM job id.
Stage 2 — Teacher training#
The teacher panel is a set of specialised RL runs that each start from the
Student RLVR output (Stage 1) (see Checkpoint flow above). Each teacher
runs independently with its own YAML config under examples/nemo_gym/nemotron-3-ultra/.
The teachers don’t depend on each other and can run in parallel.
IFBench Teacher#
RLHF teacher specializing in instruction following, abstention, and refusal
behavior. Trained at 49k context with a smaller batch (GBS=2048) and lower
learning rate (lr=2.5e-6) than the student RLVR stage.
Config: examples/nemo_gym/nemotron-3-ultra/ifbench_teacher.yaml
TP=8, EP=64, CP=8, PP=1
max_total_sequence_length=49152train_global_batch_size=2048,num_prompts_per_step=128,num_generations_per_prompt=16Learning rate
2.5e-6constantDefault cluster shape: 80 nodes (32 training + 28 vLLM + 20 Gym)
EXP_NAME=ultra-ifbench-teacher \
CONFIG_PATH=examples/nemo_gym/nemotron-3-ultra/ifbench_teacher.yaml \
MODEL_PATH=/path/to/student_rlvr_output \
TRAIN_PATH=$DATA_DIR/ifbench.train.jsonl \
VAL_PATH=$DATA_DIR/ifbench.val.jsonl \
NUM_TRAIN_NODES=32 \
NUM_GEN_NODES=28 \
NUM_GYM_NODES=20 \
ENABLE_MTP_INFERENCE=1 \
CONTAINER=/path/to/nemo-rl-container \
SANDBOX_CONTAINER=/path/to/nemo-skills-sandbox.sqsh \
PERSISTENT_CACHE=/path/to/persistent/cache \
EXTRA_MOUNTS=/lustre:/lustre \
SLURM_PARTITION=$SLURM_PARTITION \
SLURM_ACCOUNT=$SLURM_ACCOUNT \
GENRM_MODEL=nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-GenRM \
NL2BASH_JUDGE_MODEL=Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 \
SAFETY_JUDGE_MODEL=nvidia/Nemotron-Content-Safety-Reasoning-4B \
WANDB_API_KEY=$WANDB_API_KEY \
HF_HOME=/path/to/hf_cache \
HF_TOKEN=$HF_TOKEN \
bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh
RLHF Teacher#
General-purpose RLHF teacher trained against the pairwise GenRM comparison signal alone. Same training shape as the IFBench teacher (cluster, batch, learning rate, context).
Config: examples/nemo_gym/nemotron-3-ultra/rlhf_teacher.yaml
TP=8, EP=64, CP=8, PP=1
max_total_sequence_length=49152train_global_batch_size=2048,num_prompts_per_step=128,num_generations_per_prompt=16Learning rate
2.5e-6constantDefault cluster shape: 80 nodes (32 training + 28 vLLM + 20 Gym)
EXP_NAME=ultra-rlhf-teacher \
CONFIG_PATH=examples/nemo_gym/nemotron-3-ultra/rlhf_teacher.yaml \
MODEL_PATH=/path/to/student_rlvr_output \
TRAIN_PATH=$DATA_DIR/rlhf.train.jsonl \
VAL_PATH=$DATA_DIR/rlhf.val.jsonl \
NUM_TRAIN_NODES=32 \
NUM_GEN_NODES=28 \
NUM_GYM_NODES=20 \
ENABLE_MTP_INFERENCE=1 \
CONTAINER=/path/to/nemo-rl-container \
SANDBOX_CONTAINER=/path/to/nemo-skills-sandbox.sqsh \
PERSISTENT_CACHE=/path/to/persistent/cache \
EXTRA_MOUNTS=/lustre:/lustre \
SLURM_PARTITION=$SLURM_PARTITION \
SLURM_ACCOUNT=$SLURM_ACCOUNT \
GENRM_MODEL=nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-GenRM \
WANDB_API_KEY=$WANDB_API_KEY \
HF_HOME=/path/to/hf_cache \
HF_TOKEN=$HF_TOKEN \
bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh
This is the teacher referred to as NRL_CHAT_TEACHER1 / NRL_RLHF_TEACHER in
the MOPD config — it provides the genrm_simple_agent and
genrm_simple_agent_reasoning_off teacher signals.
Reasoning Teacher#
General reasoning teacher. The training data subsamples an RLVR
blend, where every prompt is graded by the equivalence_llm_judge agent
(LLM-judge equivalence over freeform short answers). The output checkpoint
serves the code_gen, ns_tools, math_with_judge,
equivalence_llm_judge, and mcqa agent slots in MOPD — one checkpoint,
many roles.
Config: examples/nemo_gym/nemotron-3-ultra/reasoning_teacher.yaml
TP=8, EP=32, CP=8, PP=1 (half the expert parallelism of Student RLVR)
max_total_sequence_length=65536train_global_batch_size=2048,num_prompts_per_step=128,num_generations_per_prompt=16Learning rate
3.0e-6constantmax_num_epochs=10— small sub-sampled dataset, multiple passes expectedDefault cluster shape: 128 nodes (64 training + 54 vLLM + 10 Gym)
EXP_NAME=ultra-reasoning-teacher \
CONFIG_PATH=examples/nemo_gym/nemotron-3-ultra/reasoning_teacher.yaml \
ENABLE_MTP_INFERENCE=1 \
MODEL_PATH=/path/to/student_rlvr_output \
TRAIN_PATH=$DATA_DIR/reasoning.train.jsonl \
VAL_PATH=$DATA_DIR/reasoning.val.jsonl \
NUM_TRAIN_NODES=64 \
NUM_GEN_NODES=54 \
NUM_GYM_NODES=10 \
CONTAINER=/path/to/nemo-rl-container \
SANDBOX_CONTAINER=/path/to/nemo-skills-sandbox.sqsh \
PERSISTENT_CACHE=/path/to/persistent/cache \
EXTRA_MOUNTS=/lustre:/lustre \
SLURM_PARTITION=$SLURM_PARTITION \
SLURM_ACCOUNT=$SLURM_ACCOUNT \
NL2BASH_JUDGE_MODEL=Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 \
WANDB_API_KEY=$WANDB_API_KEY \
HF_HOME=/path/to/hf_cache \
HF_TOKEN=$HF_TOKEN \
bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh
SWE Teacher#
Software-engineering RLVR teacher trained against code execution. The
swe_agents agent runs the policy’s candidate fixes inside apptainer (.sif)
container images for each SWE-Gym / SWE-rebench-V2 instance and rewards
the rollout based on test pass/fail.
Config: examples/nemo_gym/nemotron-3-ultra/swe_teacher.yaml
TP=8, EP=32, CP=32, PP=1 (large CP for long context)
max_total_sequence_length=196608(192k context)train_global_batch_size=512,num_prompts_per_step=32,num_generations_per_prompt=16Learning rate
3.0e-6constant, advantage clip ±100,max_num_epochs=4Default cluster shape: 128 nodes (64 training + 64 vLLM + 0 Gym)
Building the SIF images#
Each rollout runs inside a per-instance apptainer (.sif) image resolved from
${SIF_DIR} via the agent’s container_formatter. Since this recipe targets
GB200 (aarch64) and the upstream SWE images ship for x86 only, the images must
be rebuilt for ARM — one per instance — then converted to .sif. The resulting
directory is reused across runs.
The released SWE blend draws from two benchmarks:
Benchmark |
HF dataset |
|
Instances |
|---|---|---|---|
SWE-Gym |
|
|
206 |
SWE-rebench-V2 |
|
|
7,610 |
1. Prerequisites. Docker, Apptainer, and uv on the build host. Build on an
ARM64 (GB200) node so images are natively aarch64 with no emulation. You also
need a container registry to publish to: set REGISTRY to its endpoint and
docker login to it first. The build scripts push every per-instance image
there, and the conversion step (3) pulls them back to produce the .sif files —
so the registry must be reachable from both the build and the convert hosts.
Install Apptainer on the build host via the official PPA, pinned to the version
the training container ships (so the .sif format matches — see
docker/install_apptainer.sh):
sudo add-apt-repository -y ppa:apptainer/ppa
sudo apt-get update
sudo apt-get install -y "apptainer=1.5.0-2-1~$(. /etc/os-release && echo "$VERSION_CODENAME")"
Storage. The registry accumulates ~7,800 images, plus the converted
.sifset on the build host. The pushed images can be deleted once every.sifhas been built.
2. Build the per-instance images.
git clone -b ultra-v3 https://github.com/nujoug/swe-gym-arm-build
git clone -b ultra-v3 https://github.com/nujoug/swe-rebench-v2-arm-build
export REGISTRY=registry.example.com/ultra-swe # your registry endpoint (docker login first)
# --- SWE-Gym (206 images) ---
# No verify gate: this wrapper builds + pushes only. After it finishes, drop any
# instance listed under handoff/failed_instances/ before using the images.
cd swe-gym-arm-build
uv venv && source .venv/bin/activate && uv pip install -e .
python scripts/batch_build_push.py \
--dataset SWE-Gym/SWE-Gym --split train \
--instance_ids_file swe_gym_instance_ids.txt \
--registry "${REGISTRY}/swe-gym" --push_env_images \
--max_workers 8 --state_file build_push_state.json
deactivate; cd ..
# --- SWE-rebench-V2 (7,610 images) ---
# Omitting --skip-eval enables the verify gate: each image is built, the gold
# patch is applied, the tests are run, and only images whose FAIL_TO_PASS /
# PASS_TO_PASS transition matches are published (others land in
# handoff/failed_instances/).
cd swe-rebench-v2-arm-build
uv venv && source .venv/bin/activate && uv pip install -r requirements.txt
# Build the per-language base images first (Go/Java/Rust/Python/… environments
# that the instance images layer on top of).
# Note: expect one known failure when building scala_base (its Dockerfile runs `foundryup`, which 403s on the GitHub API).
python3 scripts/build_all_arm_bases.py --platform linux/arm64 --keep-going --max-workers 4 --skip-existing
python3 scripts/prepare_ready_tasks.py --hf-dataset nebius/SWE-rebench-V2 --output ready_tasks.json
python3 scripts/build_eval_cleanup.py \
--json ready_tasks.json --platform linux/arm64 --max-workers 8 \
--report-json eval_report.json --skip-done \
--gitlab-registry "${REGISTRY}/swerebenchv2"
deactivate; cd ..
3. Convert to .sif and lay out ${SIF_DIR}. build_swe_sif_images.py pulls
each published image and runs apptainer build under the exact filename the
recipe expects — SWE-Gym from the swe-gym:sweb.eval.arm64.<id> tags, and
SWE-rebench-V2 from the verified (passed_match) instances in eval_report.json.
It skips images already converted and continues past any that are missing (e.g.
instances that failed to build), recording them in ${SIF_DIR}/missing_instances.txt.
Run it from the NeMo RL repo root:
export SIF_DIR=/path/to/sif/images
python examples/nemo_gym/build_swe_sif_images.py \
--registry "${REGISTRY}" --sif-dir "${SIF_DIR}" \
--swe-gym-ids /path/to/swe-gym-arm-build/swe_gym_instance_ids.txt \
--rebench-report /path/to/swe-rebench-v2-arm-build/eval_report.json
With ${SIF_DIR} populated, launch the SWE teacher:
EXP_NAME=ultra-swe-teacher \
CONFIG_PATH=examples/nemo_gym/nemotron-3-ultra/swe_teacher.yaml \
ENABLE_MTP_INFERENCE=1 \
MODEL_PATH=/path/to/student_rlvr_output \
TRAIN_PATH=$DATA_DIR/swe.train.jsonl \
VAL_PATH=$DATA_DIR/swe.val.jsonl \
NUM_TRAIN_NODES=64 \
NUM_GEN_NODES=64 \
NUM_GYM_NODES=0 \
SIF_DIR=/path/to/sif/images \
CONTAINER=/path/to/nemo-rl-container \
SANDBOX_CONTAINER=/path/to/nemo-skills-sandbox.sqsh \
PERSISTENT_CACHE=/path/to/persistent/cache \
EXTRA_MOUNTS=/lustre:/lustre \
SLURM_PARTITION=$SLURM_PARTITION \
SLURM_ACCOUNT=$SLURM_ACCOUNT \
WANDB_API_KEY=$WANDB_API_KEY \
HF_HOME=/path/to/hf_cache \
HF_TOKEN=$HF_TOKEN \
bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh
Stage 3 — MOPD#
Multi-Teacher On-Policy Distillation. The Student RLVR output is the student; each Gym agent is routed to one of the Stage 2 teacher checkpoints. Trains the student to match per-agent teacher distributions.
Config: examples/nemo_gym/nemotron-3-ultra/mopd.yaml
TP=8, EP=64, CP=32, PP=1, max context 192k
Teacher parallelism: TP=8, CP=2, EP=16, 4 nodes per teacher
Routing: agent → teacher checkpoint baked into the YAML via
${_teachers.<role>}references; only_teachers.generalis required and every other slot falls back to it.Default cluster shape: 224 nodes (64 training + 128 vLLM + 12 Gym + 20 teachers).
Teacher mapping#
Logical slot |
Path source |
MOPD agents it serves |
|---|---|---|
|
Student RLVR output |
|
|
RLHF Teacher |
|
|
IFBench Teacher |
|
|
Reasoning Teacher |
|
|
SWE Teacher |
all |
Launch#
Pass STAGE_TYPE=mopd to the launcher to enable the teacher-pool node math
and the _teachers.X Hydra overrides. NRL_GENERAL_TEACHER_PATH is required;
the other four teacher paths are optional and fall back to general when
unset.
STAGE_TYPE=mopd \
EXP_NAME=ultra-mopd-stage1 \
CONFIG_PATH=examples/nemo_gym/nemotron-3-ultra/mopd.yaml \
ENABLE_MTP_INFERENCE=1 \
MODEL_PATH=/path/to/student_rlvr_output \
TRAIN_PATH=$DATA_DIR/mopd.train.jsonl \
VAL_PATH=$DATA_DIR/mopd.val.jsonl \
NUM_TRAIN_NODES=64 \
NUM_GEN_NODES=128 \
NUM_GYM_NODES=12 \
NUM_UNIQUE_TEACHERS=5 \
NUM_NODES_PER_TEACHER=4 \
NRL_GENERAL_TEACHER_PATH=/path/to/student_rlvr_output \
NRL_RLHF_TEACHER_PATH=/path/to/rlhf_teacher_output \
NRL_IFBENCH_TEACHER_PATH=/path/to/ifbench_teacher_output \
NRL_REASONING_TEACHER_PATH=/path/to/reasoning_teacher_output \
NRL_SWE_TEACHER_PATH=/path/to/swe_teacher_output \
CONTAINER=/path/to/nemo-rl-container \
SANDBOX_CONTAINER=/path/to/nemo-skills-sandbox.sqsh \
PERSISTENT_CACHE=/path/to/persistent/cache \
EXTRA_MOUNTS=/lustre:/lustre \
SLURM_PARTITION=$SLURM_PARTITION \
SLURM_ACCOUNT=$SLURM_ACCOUNT \
GENRM_MODEL=nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-GenRM \
NL2BASH_JUDGE_MODEL=Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 \
SAFETY_JUDGE_MODEL=nvidia/Nemotron-Content-Safety-Reasoning-4B \
WANDB_API_KEY=$WANDB_API_KEY \
HF_HOME=/path/to/hf_cache \
HF_TOKEN=$HF_TOKEN \
bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh