Skip to main content

Ctrl+K

NeMo-AutoModel

GitHub

NeMo-AutoModel

GitHub

Table of Contents

Get Started

Introduction to the NeMo Automodel Repository
Install NeMo Automodel
Run on Your Local Workstation
Run on a Cluster (Slurm / Multi-node)

Announcements

Accelerating Large-Scale Mixture-of-Experts Training in PyTorch with NeMo Automodel
Challenges in Enabling PyTorch Native Pipeline Parallelism for Hugging Face Transformer Models
Google Gemma 3n: Efficient Multimodal Fine-tuning Made Simple
Fine-tune Hugging Face Models Instantly with Day-0 Support with NVIDIA NeMo AutoModel

Recipes & E2E Examples

Recipes and E2E Examples
Supervised Fine-Tuning (SFT) and Parameter-Efficient Fine-Tuning (PEFT) with NeMo Automodel
Pretraining using Megatron-Core Datasets with NeMo Automodel
LLM Pre-Training with NeMo Automodel
Sequence Classification (SFT/PEFT) with NeMo Automodel
Fine-Tune Gemma 3 and Gemma 3n

Model Coverage

Overview
Large Language Models (LLMs)
Vision Language Models (VLMs)

Datasets

Dataset Overview: LLM and VLM Datasets in NeMo Automodel
Integrate Your Own Text Dataset
Use the ColumnMappedTextInstructionDataset
Integrate Your Own Multi-Modal Dataset

Development

Checkpointing in NeMo AutoModel
🚀 Gradient (Activation) Checkpointing in NeMo-AutoModel
Pipeline Parallelism with AutoPipeline
Knowledge Distillation with NeMo-AutoModel
FP8 Training in NeMo Automodel
API Reference
- nemo_automodel

Performance

NeMo AutoModel Performance Summary

API Reference
nemo_automodel
nemo_automodel.components
nemo_automodel.components.attention

`nemo_automodel.components.attention`#

Submodules#

nemo_automodel.components.attention.flex_attention
nemo_automodel.components.attention.utils

previous

nemo_automodel.components.datasets.utils

next

nemo_automodel.components.attention.flex_attention

On this page

Submodules

Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2025, NVIDIA Corporation.