> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/nightly/development/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# Advanced Training

## Docs

- [Checkpointing in NeMo AutoModel](https://docs.nvidia.com/nemo/automodel/nightly/development/checkpointing.md): Save and manage NeMo AutoModel checkpoints in Safetensors and PyTorch DCP formats, including consolidated and sharded layouts, PEFT adapters, asynchronous saves, and preemption handling.
- [Use Gradient (Activation) Checkpointing](https://docs.nvidia.com/nemo/automodel/nightly/development/gradient-checkpointing.md): 
- [Distributed Setup (Python API)](https://docs.nvidia.com/nemo/automodel/nightly/development/distributed-setup.md): Configure tensor, pipeline, context, and expert parallelism for NeMoAutoModel* loaders with the typed DistributedSetup object.
- [Pipeline Parallelism with AutoPipeline](https://docs.nvidia.com/nemo/automodel/nightly/development/pipeline-parallelism.md): Configure AutoPipeline for Hugging Face models, mix pipeline parallelism with other strategies, and debug pipeline stages.
- [Context-Parallel Vision Frame Sharding](https://docs.nvidia.com/nemo/automodel/nightly/development/cp-vision-frame-sharding.md): Reduce duplicated vision-tower work by distributing image and video frames across context-parallel ranks.
- [FP8 Training in NeMo AutoModel](https://docs.nvidia.com/nemo/automodel/nightly/development/fp8-training.md): 
- [Mixed-Precision Training](https://docs.nvidia.com/nemo/automodel/nightly/development/mixed-precision-training.md): Recommended dtype patterns for fp32 master weights and optimizer state, bf16 compute, and FSDP2 mixed precision in NeMo AutoModel.
- [MLflow Logging in NeMo AutoModel](https://docs.nvidia.com/nemo/automodel/nightly/development/mlflow-logging.md): 
- [Trackio Logging in NeMo AutoModel](https://docs.nvidia.com/nemo/automodel/nightly/development/trackio-logging.md): Track NeMo AutoModel training runs locally or on a Hugging Face Space with Trackio.
- [SwanLab Logging in NeMo AutoModel](https://docs.nvidia.com/nemo/automodel/nightly/development/swanlab-logging.md): Log NeMo AutoModel training runs to SwanLab alongside or instead of Weights & Biases, including offline clusters.
- [Breaking Changes](https://docs.nvidia.com/nemo/automodel/nightly/development/breaking-changes.md):