Hugging Face Transformers API Compatibility

NeMo AutoModel is built to work with the Hugging Face ecosystem. In practice, compatibility comes in two layers:

• API compatibility: For many workflows, you can keep your existing transformers code and swap in NeMo AutoModel drop-in wrappers (NeMoAutoModel*, NeMoAutoTokenizer) with minimal changes.
• Artifact compatibility: NeMo AutoModel produces Hugging Face-compatible checkpoints (config, tokenizer, and safetensors) that can be loaded by Hugging Face Transformers and downstream tools such as vLLM and SGLang.

This page summarizes Hugging Face compatibility in NeMo AutoModel, outlines key differences, and provides side-by-side examples.

Transformers Version Compatibility

NeMo AutoModel defaults to Hugging Face Transformers v5. Saved checkpoints and tokenizer artifacts target Transformers v5 and later unless you opt into an experimental compatibility path.

Transformers v5 Support

NeMo AutoModel currently pins Hugging Face Transformers to the v5 major line. Refer to pyproject.toml for the exact version.

This means:

• NeMo AutoModel is primarily tested and released against Transformers v5.x.
• If a new model on the Hugging Face Hub requires a newer version of the transformers library than the pinned version, you must upgrade NeMo AutoModel to a version that supports the newer release.

Transformers v4 Interoperability

Some downstream environments may still run Transformers v4. NeMo AutoModel development, tests, and default saved artifacts target Transformers v5 and later.

NeMo AutoModel keeps v4 interoperability where practical:

• Compatibility shims: Apply small compatibility patches automatically through built-in recipes to smooth over known API differences (for example, cache utility method names).
• Code backports: Vendor or backport Hugging Face code for specific model families to allow users to run models even if their upstream integration has changed between major Transformers releases.
• Artifact format: Write checkpoints in Hugging Face-compatible save_pretrained layouts (config, tokenizer, and safetensors). Default exports target Transformers v5 and later.

Drop-In Compatibility and Key Differences

NeMo AutoModel matches the core loading and execution patterns of Hugging Face Transformers while introducing performance enhancements and distributed training features.

Drop-In Support

These APIs follow familiar Hugging Face Transformers patterns for model loading, configuration, tokenization, and generation.

• Model loading: Load by model ID or local path using from_pretrained(...).
• Configuration: Use standard Hugging Face config objects such as AutoConfig and config.json.
• Tokenizers: Use standard PreTrainedTokenizerBase behavior, including __call__ to create tensors, decode, and batch_decode.
• Generation: Use model.generate(...) and standard generation kwargs.

Key Differences

NeMo AutoModel introduces several optimizations and architectural changes to improve training efficiency and hardware utilization.

• Performance features: NeMo AutoModel can automatically apply optional kernel patches and optimizations (for example, SDPA selection, Liger kernels, and DeepEP) while keeping the public model API the same.
• Distributed training stack: NeMo AutoModel recipes and CLI are designed for multi-GPU and multi-node fine-tuning with PyTorch-native distributed features such as FSDP2 and pipeline parallelism. When loading models directly from Python, pass a DistributedSetup using from_pretrained(..., distributed_setup=...) to enable tensor, pipeline, context, and expert parallelism.
• CUDA expectation: NeMo AutoModel NeMoAutoModel* wrappers are primarily optimized for NVIDIA GPU workflows and offer support for CPU workflows as well.

API Mapping

The following sections map standard Hugging Face classes and methods to their NeMo AutoModel equivalents.

Class and Method Mapping

Side-by-Side Examples

These comparative examples demonstrate how to transition standard Hugging Face code to NeMo AutoModel.

Load Models and Tokenizers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "gpt2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
)

import torch
from nemo_automodel import NeMoAutoModelForCausalLM, NeMoAutoTokenizer

model_id = "gpt2"

tokenizer = NeMoAutoTokenizer.from_pretrained(model_id)
model = NeMoAutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
)

Generate Text

This snippet assumes you already have a model and tokenizer (see the loading snippet above).

import torch

prompt = "Write a haiku about GPU kernels."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=64)

print(tokenizer.decode(out[0], skip_special_tokens=True))

import torch

prompt = "Write a haiku about GPU kernels."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=64)

print(tokenizer.decode(out[0], skip_special_tokens=True))

Compare Tokenizers

NeMo AutoModel provides NeMoAutoTokenizer as a Transformers-like auto-tokenizer with a small registry for specialized backends (and a safe fallback when no specialization is needed).

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")

from nemo_automodel import NeMoAutoTokenizer

tok = NeMoAutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")

Select the loading route with tokenizer_backend:

• "nemo_auto" (the default) uses NeMo AutoModel’s model-type registry and falls back to its Transformers-compatible wrapper.
• "nemo_wrapped_auto" bypasses registered NeMo tokenizers and uses Transformers AutoTokenizer with NeMo AutoModel’s tokenizer compatibility wrapper.
• "transformers_auto" calls Transformers AutoTokenizer directly.
• "tokenizers" loads tokenizer.json directly through Transformers TokenizersBackend. Use this route when the serialized tokenizer should be authoritative and no repository-provided tokenizer code is needed. Tokenizer policy arguments such as add_bos_token, add_eos_token, split_special_tokens, fix_mistral_regex, and padding_side are forwarded to that backend.

For example:

tok = NeMoAutoTokenizer.from_pretrained(
    "mistralai/Ministral-3-3B-Instruct-2512-BF16",
    tokenizer_backend="tokenizers",
    fix_mistral_regex=True,
    split_special_tokens=True,
    add_bos_token=True,
    add_eos_token=False,
)

The legacy force_default=True option is equivalent to tokenizer_backend="nemo_wrapped_auto", and force_hf=True is equivalent to tokenizer_backend="transformers_auto". Equivalent legacy flags and explicit routes can be combined. Contradictory route selections raise ValueError.

Save Checkpoints in NeMo AutoModel and Load Everywhere

NeMo AutoModel training recipes write checkpoints as sharded safetensors by default and generate a per-checkpoint helper that can export Hugging Face-compatible consolidated safetensors after training:

• See Checkpointing for checkpoint formats and example directory layouts.
• See Model Coverage for notes on how model support depends on the pinned Transformers version.

If your goal is to train or fine-tune in NeMo AutoModel and deploy in the Hugging Face ecosystem, keep model_save_format: safetensors. Use either save_consolidated: final for final-checkpoint export or save_consolidated: false plus bash <checkpoint>/model/consolidate.sh after training. Then load model/consolidated/ using standard Hugging Face APIs or downstream inference engines. Set save_consolidated: every (or legacy true) only if you want inline Hugging Face export at every checkpoint save.