> This page is for version 0.7.0 · 26.10. > For other versions, use one of these documentation indexes: > - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt > - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt > - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt > - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt > - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server. # Model Coverage Overview > Hugging Face Auto class support, release history, day-0 coverage, and custom architecture registration for NeMo AutoModel. NeMo AutoModel integrates with Hugging Face `transformers`. Any LLM or VLM that can be instantiated through `transformers` can also be used with NeMo AutoModel, subject to runtime, third-party software dependencies, and feature compatibility. ## Supported Hugging Face Auto Classes | Auto Class | Task | Status | Details | | ------------------------------------ | ------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AutoModelForCausalLM` | Text Generation (LLM) | Supported | See [LLM model list](/nemo/automodel/v0.7/model-coverage/large-language-models/overview) | | `AutoModelForSeq2SeqLM` | Text-to-Text (Encoder-Decoder) | Supported | See the [HF compatibility mapping](/nemo/automodel/v0.7/get-started/hf-compatibility#class-and-method-mapping) and [T5 example recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/r0.7.0/examples/llm_finetune/t5/t5_small_squad.yaml) | | Block-Diffusion LLMs | Text Generation (Diffusion LLM) | Supported | See [Diffusion LLM model list](/nemo/automodel/v0.7/model-coverage/dllm/overview) | | `AutoModelForImageTextToText` | Image-Text-to-Text (VLM) | Supported | See [VLM model list](/nemo/automodel/v0.7/model-coverage/vision-language-models/overview) | | Custom multimodal models | Unified multimodal training | Supported | See [Multimodal model list](/nemo/automodel/v0.7/model-coverage/multimodal/overview) | | `AutoModelForSequenceClassification` | Sequence Classification | Work in Progress | Early support; interfaces might change | | Diffusers Pipelines | Diffusion Generation (T2I, T2V) | Supported | See [Diffusion model list](/nemo/automodel/v0.7/model-coverage/diffusion/overview) | | `NeMoAutoModelBiEncoder` | Embedding Models | Supported | See [Embedding model list](/nemo/automodel/v0.7/model-coverage/embedding-models/overview) | | `NeMoAutoModelCrossEncoder` | Reranking Models | Supported | See [Reranking model list](/nemo/automodel/v0.7/model-coverage/reranking-models/overview) | ## Release Log The table summarizes selected model and feature additions in published NeMo AutoModel releases. Dates are the GitHub release publication dates; each version links to its full release notes. For model additions on `main`, including changes after the latest release, see the [Model Support Log](/nemo/automodel/v0.7/whats-new/model-release-log). For tested architectures and example configs, see the [LLM](/nemo/automodel/v0.7/model-coverage/large-language-models/overview), [VLM](/nemo/automodel/v0.7/model-coverage/vision-language-models/overview), and [Multimodal](/nemo/automodel/v0.7/model-coverage/multimodal/overview) pages. | Release | Published | New or Expanded Model Support | Key Features | | -------------------------------------------------------------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [**0.6.0**](https://github.com/NVIDIA-NeMo/Automodel/releases/tag/v0.6.0) | August 26, 2026 | Kimi K3, GLM-5.2, Laguna S 2.1, MiniMax-M3 VL, Inkling, LTX-2.3, Qwen-Image-Edit-2511, DiffusionGemma | Packed variable-length context parallelism, vision-tower frame sharding, DSpark and ViSpec draft training, preemption checkpointing, diffusion context parallelism | | [**0.5.0**](https://github.com/NVIDIA-NeMo/Automodel/releases/tag/v0.5.0) | July 2, 2026 | DeepSeek-V4 Flash, MiMo-V2-Flash, Hy3-preview, MiniMax-M2.7, Qwen3.6 VLM, Step-3.7-Flash, Bagel, FLUX.2-dev, Qwen-Image, Wan 2.2, LLaDA2, Nemotron-Labs-Diffusion | Agent SFT, EAGLE/P-EAGLE/DFlash draft training, MXFP8 MoE training on GB200, VLM QLoRA and knowledge distillation, late-interaction retrieval | | [**0.4.0**](https://github.com/NVIDIA-NeMo/Automodel/releases/tag/v0.4.0) | April 28, 2026 | GLM-5/5.1, MiniMax-M2.5, Gemma 4, Mistral Small 4, Qwen3.5 VLM, FLUX, HunyuanVideo, Wan 2.1, LLaDA | Diffusion training and LoRA, discrete-diffusion LLM SFT, cross-encoder/reranker training, knowledge distillation with TP and PP, HybridEP/UCCL-EP, SkyPilot launcher | | [**0.3.0**](https://github.com/NVIDIA-NeMo/Automodel/releases/tag/v0.3.0) | March 2, 2026 | DeepSeek V3.2, Step-3.5-Flash, MiniMax M2, Nemotron-3-Nano-30B, GLM-4.7/4.7-Flash, Devstral-Small-2, FunctionGemma, Ministral3, Kimi-VL/Kimi-K2.5 VL, Nemotron-Parse, Qwen3-VL-MoE, Llama-Embed-Nemotron-8B | Transformers v5, MoE LoRA and DoRA, VLM pipeline parallelism, bi-encoder training and hard-negative mining, streaming SafeTensors checkpoint saving, Dion optimizer | | [**0.2.0**](https://github.com/NVIDIA-NeMo/Automodel/releases/tag/v0.2.0) | December 4, 2025 | GPT-OSS 20B/120B, Qwen3-Next/235B, GLM-4.5/4.6, Qwen2.5-VL, Qwen3-VL, Qwen3-Omni, InternVL | Tool calling, streaming datasets, QAT for SFT, sequence classification, async DCP checkpointing, MLflow, CP and sequence packing for MoE | | [**0.1.2**](https://github.com/NVIDIA-NeMo/Automodel/releases/tag/v0.1.2) | October 23, 2025 | No new models listed | ColumnMappedDataset sample limits; step-scheduler fixes for zero-based indexing and gradient accumulation | | [**0.1.0**](https://github.com/NVIDIA-NeMo/Automodel/releases/tag/v0.1.0) | October 8, 2025 | DeepSeek V3 optimized implementation | Pretraining, knowledge distillation, FP8 with torchao, HSDP, automatic pipelining, ColumnMappedDataset, MegatronFSDP, packed sequences, Triton LoRA kernels | | [**0.1.0rc0**](https://github.com/NVIDIA-NeMo/Automodel/releases/tag/v0.1.0rc0) (prerelease) | September 17, 2025 | Initial prerelease | First release candidate | ## Day-0 Support * NeMo AutoModel closely tracks the latest `transformers` version and updates its dependency regularly. * New models released on the Hugging Face Hub might require the latest `transformers` version, necessitating a package upgrade. * The team is developing a CI pipeline that automatically updates the supported `transformers` version when a new release is detected, enabling faster day-0 support. ## Custom Model Registry NeMo AutoModel includes a custom model registry that allows teams to: * Add custom implementations to extend support to models not yet covered upstream. * Provide optimized or faster implementations for specific models while retaining the same NeMo AutoModel interface. ### Register an Architecture The registry matches an architecture name against the first value in the checkpoint's `config.json` `architectures` list. The name is case-sensitive. The registered class must be a `torch.nn.Module` class that is compatible with the selected `NeMoAutoModel*` loader and accepts the resolved Hugging Face config as its first constructor argument. #### Register in Python Call `register_architecture` before constructing or loading the model: ```python from nemo_automodel import NeMoAutoModelForCausalLM, register_architecture from my_package.models import MyModelForCausalLM register_architecture("MyModelForCausalLM", MyModelForCausalLM) model = NeMoAutoModelForCausalLM.from_pretrained("my-org/my-model") ``` Registering a built-in or previously registered name raises `ValueError`, even when the same class is registered again. Pass `exist_ok=True` only when you intentionally want to replace the existing model class. #### Register from an Installed Package An installed package can advertise model classes without requiring application startup code. Add an entry point for each architecture to the package's `pyproject.toml`: ```toml [project.entry-points."nemo_automodel.architectures"] MyModelForCausalLM = "my_package.models:MyModelForCausalLM" ``` The entry-point name is the architecture name, and its value must use the `module.path:ClassName` format. NeMo AutoModel discovers these entry points when its model registry initializes and imports the target module only when it resolves that architecture. Install the package before starting the Python process. If an entry-point name conflicts with a built-in or another discovered architecture, NeMo AutoModel skips it and logs a warning; use `register_architecture(..., exist_ok=True)` in application code for an intentional override. > **Note** > > Architecture registration selects a model implementation after the Hugging Face config is resolved. It does not > register a new `model_type`, so the checkpoint config must already be loadable by the installed versions of > Hugging Face `transformers` or NeMo AutoModel. ### Ready-to-Run Architectures The following table lists architectures represented by the ready-to-run YAML recipes in this repository. It includes both NeMo-native implementations and models that use the standard Hugging Face implementation path. This is a practical starting set rather than an exhaustive compatibility list. NeMo AutoModel can also work with additional models supported by the installed version of the Hugging Face `transformers` library, although models without a checked-in recipe might require some configuration for a particular training setup. ## Having Issues? If a model from the Hugging Face Hub does not work as expected, see [Troubleshooting](/nemo/automodel/v0.7/model-coverage/troubleshooting) for common issues and solutions.