Training Configuration
Want to learn about training concepts at a high level? Check out the Customization concepts page.
NeMo Customizer ships two training backends, and each accepts its own job configuration. Choose the backend that matches your hardware and training goal, then configure the hyperparameters from that backend’s schema below.
The two backends do not share field names. For example, Automodel uses batch.global_batch_size / batch.micro_batch_size and a parallelism block; Unsloth uses batch.per_device_train_batch_size / batch.gradient_accumulation_steps and a hardware block. Both schemas reject unknown keys, so a field from one backend will not validate against the other.
Each backend can also print its live schema, and the generated REST shapes are in the Customizer API Reference (search for AutomodelJobInput and UnslothJobInput).
Start from a Reference Recipe
The training libraries publish benchmarked recipes per model. Start a job from the closest one instead of from defaults:
- Automodel jobs: Automodel
r0.7.0LLM fine-tuning recipes. Nemotron recipes are underllm_finetune/nemotron. - RL jobs (DPO, GRPO): NeMo-RL
r0.8.0recipes. Only thefsdp2andautomodelrecipes apply; Megatron recipes do not.
Copy the learning rate, LoRA rank and alpha, batch sizes, sequence length, and parallel layout into the job, then scale the batch to your GPU count. Ignore max_steps and the ci block; those are CI smoke-test settings.
Automodel Configuration
An Automodel job is configured with the following top-level sections: model, dataset, training, schedule, batch, optimizer, parallelism, output, and (optionally) integrations.
Model and Dataset
Training Method
When to set training.tokenizer_backend:
- Usually never. For every Nemotron and Qwen checkpoint Customizer documents, the unset default loads the right tokenizer.
- Nemotron 3 Embed 1B declares model type
ministral3, which Automodel maps to Mistral’stekken.json, but the checkpoint ships onlytokenizer.json. The retrieval default handles this, so never setnemo_autofor it. - SFT or distillation on such a checkpoint: set
nemo_wrapped_auto. - Mistral-family LLMs that ship
tekken.json: keep the default.
When a full-weight distillation job leaves training.precision unset and uses Adam or AdamW, the student is stored in fp32. Those optimizers keep no separate fp32 copy of the weights, so bf16 storage would make every update bf16.
LoRA parameters (training.lora):
Schedule
Batch
Optimizer
Retrieval recipes replace the batch and optimizer defaults above. When training.recipe resolves to bi_encoder or cross_encoder, either set explicitly or chosen by auto from the checkpoint head, each of these fields you leave unset takes the recipe’s value:
auto reads the checkpoint head that model registration infers from the checkpoint’s files. A Qwen3 reranker (a Qwen3ForCausalLM that scores with its yes/no logits) looks like a chat model unless its model card sets pipeline_tag: text-ranking, as the published Qwen3-Reranker does. A Qwen3 reranker trained by Customizer, or fine-tuned elsewhere, doesn’t carry that tag, so auto trains it with sft. Set training.recipe: cross_encoder explicitly to re-tune it.
Parallelism
The parallelism block scales Automodel training across GPUs and nodes.
GPU relationships and constraints:
total_gpus = num_gpus_per_node × num_nodes.total_gpusmust be divisible bytensor_parallel_size × pipeline_parallel_size × context_parallel_size.data_parallel_sizeis derived astotal_gpus / (TP × PP × CP), andglobal_batch_sizemust be divisible bymicro_batch_size × data_parallel_size.- For MoE models, when
expert_parallel_sizeis set: the number of experts must be divisible byexpert_parallel_size,(data_parallel_size × context_parallel_size)must be divisible byexpert_parallel_size, andtensor_parallel_sizemust be1whenexpert_parallel_size > 1.
Distillation
When training.training_type is "distillation", the following additional fields configure knowledge distillation from a teacher model:
- Knowledge distillation uses logit-pair distillation — the student learns to match the teacher’s output probability distribution.
- Both student and teacher must be full-weight Model Entities and share the same tokenizer and vocabulary. Use models from the same family (for example, Qwen3 0.6B + Qwen3 8B).
- Both models are loaded during training; plan GPU memory accordingly (or set
offload_teacher).
Unsloth Configuration
An Unsloth job is configured with the following top-level sections: model, dataset, training, schedule, batch, optimizer, hardware, output, and (optionally) integrations. Unsloth runs on a single GPU and supports 4-bit / 8-bit quantized loading for LoRA. Full-weight training must load the model without quantization.
Model
Full-weight training (training.finetuning_type: "all_weights") cannot be combined with quantized loading. Set load_in_4bit and load_in_8bit to false for full-weight runs.
Dataset
Training Method
LoRA parameters (training.lora):
Schedule
Batch
Optimizer
Hardware
Output
The output.save_method field accepts:
The merged_* methods are only valid when training.finetuning_type is lora. When training.finetuning_type is all_weights, omit output.save_method; the training driver saves the full trained checkpoint.
Reinforcement Learning Configuration (DPO and GRPO)
The rl backend runs two methods on a Ray cluster, selected by training.type. It requires a platform configured with platform.runtime: kubernetes and has no local Docker fallback.
training.type is required. It is the union discriminator, so a payload that omits it is rejected with union_tag_not_found rather than defaulting to one of the two.
Unlike the other backends, model, dataset, and (for GRPO) environment are plain string references rather than objects:
Shared Fields
DPO Fields
DPO is full-weight only.
GRPO Fields
Advanced controls, collapsed by default in most workflows:
Parameter Efficiency (GRPO)
DPO is full-weight only. GRPO trains either way, selected by training.finetuning_type.
The output type is inferred from this field; output carries only name.
Leaving both target_modules and exclude_modules unset adapts every linear layer. Setting either one turns that off, so exclude_modules alone does not mean “all linear layers except these.” Supplying a lora block alongside all_weights is rejected.
GRPO progress is read on reward, not loss — the surrogate loss oscillates near zero and says nothing about run quality. See GRPO and Reward Environments for environment packaging, dataset format, and the metrics to monitor.
GPU Memory Guidelines
Estimated GPU requirements by model size:
Use LoRA for most fine-tuning tasks — it is significantly more memory-efficient and often achieves results comparable to full fine-tuning. On a single memory-constrained GPU, the Unsloth backend with 4-bit loading fits the largest adapters.