Skip to navigation
Model CoverageDiffusion Language ModelsDiffusion Language Models (dLLMs)

Diffusion Language Models (dLLMs)

View as MarkdownOpen in Claude

Diffusion language models (dLLMs) generate text by denoising rather than by using left-to-right autoregression. A fixed-length response “canvas” is corrupted and then iteratively refined, so tokens are produced in parallel and can be revised across steps. NeMo AutoModel supports fine-tuning block-diffusion dLLMs with the same recipe-driven FSDP2 and expert parallel training stack used for LLMs and VLMs.

Supported Models

LLaDA, LLaDA2, Nemotron-Labs-Diffusion, and DFlash load through their Hugging Face remote-code implementations. DiffusionGemma has a native NeMo AutoModel implementation, and I-DLM converts a supported autoregressive checkpoint with a diffusion training objective.

This table combines recipe-backed checkpoints with documented model families. Dates show when the current checkpoint first appeared in a recipe, or when a documentation-only family page was added. See the combined model support log for recipe-backed checkpoints of every model type.

DateModelArchitecturesRecipe
2026-07-30Qwen3-8BQwen3ForCausalLM with the I-DLM objectiverecipe
2026-06-10Diffusiongemma-26B-A4B-itrecipe
2026-06-03LLaDA2.1-miniLLaDA2MoeModelLM with Hugging Face trust_remote_coderecipe
2026-06-03Qwen3-4B-DFlash-b16DFlashDraftModel with Hugging Face trust_remote_coderecipe
2026-05-23Nemotron-Labs-Diffusion-8B-BaseNemotronLabsDiffusionModel with Hugging Face trust_remote_coderecipe
2026-04-04LLaDA-8B-BaseLLaDAModelLM with Hugging Face trust_remote_coderecipe

Fine-Tuning

The dLLM Fine-Tuning Guide provides the complete recipe walkthrough for all four families. The DiffusionGemma Fine-Tuning Guide describes the block-diffusion training objective (uniform random token corruption without [MASK]), self-conditioning, and the supported feature set (SFT, LoRA, expert parallelism, and activation checkpointing).