Hugging Face Transformers API Compatibility

NeMo AutoModel is built to work with the Hugging Face ecosystem. In practice, compatibility comes in two layers:

• API compatibility: for many workflows, you can keep your existing transformers code and swap in NeMo AutoModel “drop-in” wrappers (NeMoAutoModel*, NeMoAutoTokenizer) with minimal changes.
• Artifact compatibility: NeMo AutoModel produces Hugging Face-compatible checkpoints (config, tokenizer, and safetensors) that can be loaded by Hugging Face Transformers and downstream tools (vLLM, SGLang, etc.).

This page summarizes Hugging Face compatibility in NeMo AutoModel, outlines key differences, and provides side-by-side examples.

Transformers Version Compatibility

NeMo AutoModel defaults to Hugging Face Transformers v5 while maintaining interoperability with v4 environments to balance access to new models with downstream stability.

Transformers v5 Support

NeMo AutoModel currently pins Hugging Face Transformers to the v5 major line (see pyproject.toml, currently transformers==5.5.0).

This means:

• NeMo AutoModel is primarily tested and released against Transformers v5.x.
• If a new model on the Hugging Face Hub requires a newer version of the transformers library than the pinned version, you must upgrade NeMo AutoModel to a version that supports the newer release.

Transformers v4 Interoperability

Some downstream environments may still run Transformers v4, while NeMo AutoModel development and tests now target Transformers v5.

NeMo AutoModel keeps v4 interoperability where practical:

• Compatibility shims: Apply small compatibility patches automatically through built-in recipes to smooth over known API differences (for example, cache utility method names).
• Code backports: Vendor or backport Hugging Face code for specific model families to allow users to run models even if their upstream integration has changed between major Transformers releases.
• Stable artifact format: Write checkpoints in Hugging Face-compatible save_pretrained layouts (config, tokenizer, and safetensors) for standard loading APIs and non-Transformers tools.

Drop-In Compatibility and Key Differences

NeMo AutoModel matches the core loading and execution patterns of Hugging Face Transformers while introducing performance enhancements and distributed training features.

Drop-In Support

These APIs follow familiar Hugging Face Transformers patterns for model loading, configuration, tokenization, and generation.

• Model loading: Load by model ID or local path using from_pretrained(...).
• Configuration: Use standard Hugging Face config objects such as AutoConfig and config.json.
• Tokenizers: Use standard PreTrainedTokenizerBase behavior, including __call__ to create tensors, decode, and batch_decode.
• Generation: Use model.generate(...) and standard generation kwargs.

Key Differences

NeMo AutoModel introduces several optimizations and architectural changes to improve training efficiency and hardware utilization.

• Performance features: NeMo AutoModel can automatically apply optional kernel patches and optimizations (e.g., SDPA selection, Liger kernels, DeepEP, etc.) while keeping the public model API the same.
• Distributed training stack: NeMo AutoModel recipes and CLI are designed for multi-GPU and multi-node fine-tuning with PyTorch-native distributed features (FSDP2, pipeline parallelism, etc.). When loading models directly from Python, pass a DistributedSetup via from_pretrained(..., distributed_setup=...) to enable tensor, pipeline, context, and expert parallelism.
• CUDA expectation: NeMo AutoModel NeMoAutoModel* wrappers are primarily optimized for NVIDIA GPU workflows and offer support for CPU workflows as well.

API Mapping

The following sections map standard Hugging Face classes and methods to their NeMo AutoModel equivalents.

Class and Method Mapping

Side-by-Side Examples

These comparative examples demonstrate how to transition standard Hugging Face code to NeMo AutoModel.

Load Models and Tokenizers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "gpt2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
)

import torch
from nemo_automodel import NeMoAutoModelForCausalLM, NeMoAutoTokenizer

model_id = "gpt2"

tokenizer = NeMoAutoTokenizer.from_pretrained(model_id)
model = NeMoAutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
)

Generate Text

This snippet assumes you already have a model and tokenizer (see the loading snippet above).

import torch

prompt = "Write a haiku about GPU kernels."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=64)

print(tokenizer.decode(out[0], skip_special_tokens=True))

import torch

prompt = "Write a haiku about GPU kernels."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=64)

print(tokenizer.decode(out[0], skip_special_tokens=True))

Compare Tokenizers

NeMo AutoModel provides NeMoAutoTokenizer as a Transformers-like auto-tokenizer with a small registry for specialized backends (and a safe fallback when no specialization is needed).

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")

from nemo_automodel import NeMoAutoTokenizer

tok = NeMoAutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")

Save Checkpoints in NeMo AutoModel and Load Everywhere

NeMo AutoModel training recipes write checkpoints as sharded safetensors by default and generate a per-checkpoint helper that can export Hugging Face-compatible consolidated safetensors after training:

• See Checkpointing for checkpoint formats and example directory layouts.
• See Model Coverage for notes on how model support depends on the pinned Transformers version.

If your goal is to train or fine-tune in NeMo AutoModel and deploy in the HF ecosystem, the recommended workflow is to keep model_save_format: safetensors with either save_consolidated: final for final-checkpoint export or save_consolidated: false plus bash <checkpoint>/model/consolidate.sh after training. Then, load model/consolidated/ using standard HF APIs or downstream inference engines. Set save_consolidated: every (or legacy true) only if you want inline HF export at every checkpoint save.