Qwen3-8B

The I-DLM recipe converts the autoregressive Qwen/Qwen3-8B checkpoint into a diffusion language model using an all-masked block-diffusion objective.

Fine-Tune Qwen3-8B

From the repository root, run:

uv run torchrun --nproc-per-node=8 nemo_automodel/recipes/dllm/train_ft.py \
  -c examples/dllm_sft/qwen3_8b_idlm.yaml

Model Reference

Model Architecture

Example Model and Recipe

See the dLLM Fine-Tuning Guide.