> This page is for version 0.7.0 · 26.10.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# Hugging Face Transformers API Compatibility

> Summarizes Hugging Face Transformers API and artifact compatibility in NeMo AutoModel, including version support, drop-in APIs, and side-by-side examples.

NeMo AutoModel is built to work with the Hugging Face ecosystem.
In practice, compatibility comes in two layers:

* **API compatibility**: For many workflows, you can keep your existing `transformers` code and swap in NeMo AutoModel drop-in wrappers (`NeMoAutoModel*`, `NeMoAutoTokenizer`) with minimal changes.
* **Artifact compatibility**: NeMo AutoModel produces Hugging Face-compatible checkpoints (config, tokenizer, and safetensors) that can be loaded by Hugging Face Transformers and downstream tools such as vLLM and SGLang.

This page summarizes Hugging Face compatibility in NeMo AutoModel, outlines key differences, and provides side-by-side examples.

## Transformers Version Compatibility

NeMo AutoModel defaults to Hugging Face Transformers v5. Saved checkpoints and tokenizer artifacts target Transformers v5 and later unless you opt into an experimental compatibility path.

### Transformers v5 Support

NeMo AutoModel currently pins Hugging Face Transformers to the v5 major line. Refer to `pyproject.toml` for the exact version.

This means:

* NeMo AutoModel is primarily tested and released against Transformers v5.x.
* If a new model on the Hugging Face Hub requires a newer version of the `transformers` library than the pinned version, you must upgrade NeMo AutoModel to a version that supports the newer release.

### Transformers v4 Interoperability

Some downstream environments may still run Transformers v4. NeMo AutoModel development, tests, and default saved artifacts target Transformers v5 and later.

NeMo AutoModel keeps v4 interoperability where practical:

* **Compatibility shims**: Apply small compatibility patches automatically through built-in recipes to smooth over known API differences (for example, cache utility method names).
* **Code backports**: Vendor or backport Hugging Face code for specific model families to allow users to run models even if their upstream integration has changed between major Transformers releases.
* **Artifact format**: Write checkpoints in Hugging Face-compatible `save_pretrained` layouts (config, tokenizer, and safetensors). Default exports target Transformers v5 and later.

> **Note**
>
> The experimental `checkpoint.v4_compatible` option preserves selected v4 metadata, but compatibility remains model- and artifact-specific. Validate that path before using a consolidated checkpoint in a v4 environment. For checkpoint layouts, refer to [Checkpointing](/nemo/automodel/v0.7/development/checkpointing).

## Drop-In Compatibility and Key Differences

NeMo AutoModel matches the core loading and execution patterns of Hugging Face Transformers while introducing performance enhancements and distributed training features.

### Drop-In Support

These APIs follow familiar Hugging Face Transformers patterns for model loading, configuration, tokenization, and generation.

* **Model loading**: Load by model ID or local path using `from_pretrained(...)`.
* **Configuration**: Use standard Hugging Face config objects such as `AutoConfig` and `config.json`.
* **Tokenizers**: Use standard `PreTrainedTokenizerBase` behavior, including `__call__` to create tensors, `decode`, and `batch_decode`.
* **Generation**: Use `model.generate(...)` and standard generation kwargs.

### Key Differences

NeMo AutoModel introduces several optimizations and architectural changes to improve training efficiency and hardware utilization.

* **Performance features**: NeMo AutoModel can automatically apply optional kernel patches and optimizations (for example, SDPA selection, Liger kernels, and DeepEP) while keeping the public model API the same.
* **Distributed training stack**: NeMo AutoModel recipes and CLI are designed for multi-GPU and multi-node fine-tuning with PyTorch-native distributed features such as FSDP2 and pipeline parallelism. When loading models directly from Python, pass a [`DistributedSetup`](/nemo/automodel/v0.7/development/distributed-setup) using `from_pretrained(..., distributed_setup=...)` to enable tensor, pipeline, context, and expert parallelism.
* **CUDA expectation**: NeMo AutoModel `NeMoAutoModel*` wrappers are primarily optimized for NVIDIA GPU workflows and offer support for CPU workflows as well.

> **Info**
>
> `NeMoAutoModelForCausalLM.from_pretrained(...)` currently assumes CUDA is available (it uses `torch.cuda.current_device()` internally). If you need CPU-only inference, use Hugging Face `transformers` directly.

## API Mapping

The following sections map standard Hugging Face classes and methods to their NeMo AutoModel equivalents.

### Class and Method Mapping

<table>
  <thead>
    <tr>
      <th>
        Hugging Face (

        `transformers`

        )
      </th>

      <th>
        NeMo AutoModel (

        `nemo_automodel`

        )
      </th>

      <th>
        Status
      </th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>
        `transformers.AutoModelForCausalLM`
      </td>

      <td>
        `nemo_automodel.NeMoAutoModelForCausalLM`
      </td>

      <td>
        ✅
      </td>
    </tr>

    <tr>
      <td>
        `transformers.AutoModelForImageTextToText`
      </td>

      <td>
        `nemo_automodel.NeMoAutoModelForImageTextToText`
      </td>

      <td>
        ✅
      </td>
    </tr>

    <tr>
      <td>
        `transformers.AutoModelForSequenceClassification`
      </td>

      <td>
        `nemo_automodel.NeMoAutoModelForSequenceClassification`
      </td>

      <td>
        ✅
      </td>
    </tr>

    <tr>
      <td>
        `transformers.AutoModelForTextToWaveform`
      </td>

      <td>
        `nemo_automodel.NeMoAutoModelForTextToWaveform`
      </td>

      <td>
        ✅
      </td>
    </tr>

    <tr>
      <td>
        `transformers.AutoModelForSeq2SeqLM`
      </td>

      <td>
        `nemo_automodel.NeMoAutoModelForSeq2SeqLM`
      </td>

      <td>
        ✅
      </td>
    </tr>

    <tr>
      <td>
        `transformers.AutoTokenizer.from_pretrained(...)`
      </td>

      <td>
        `nemo_automodel.NeMoAutoTokenizer.from_pretrained(...)`
      </td>

      <td>
        ✅
      </td>
    </tr>

    <tr>
      <td>
        `model.generate(...)`
      </td>

      <td>
        `model.generate(...)`
      </td>

      <td>
        🚧
      </td>
    </tr>

    <tr>
      <td>
        `model.save_pretrained(path)`
      </td>

      <td>
        `model.save_pretrained(path, checkpointer=...)`
      </td>

      <td>
        🚧
      </td>
    </tr>
  </tbody>
</table>

## Side-by-Side Examples

These comparative examples demonstrate how to transition standard Hugging Face code to NeMo AutoModel.

### Load Models and Tokenizers

<table>
  <thead>
    <tr>
      <th>
        Hugging Face (

        `transformers`

        )
      </th>

      <th>
        NeMo AutoModel (

        `nemo_automodel`

        )
      </th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>
        <pre>
          <code />
        </pre>
      </td>

      <td>
        <pre>
          <code />
        </pre>
      </td>
    </tr>
  </tbody>
</table>

### Generate Text

This snippet assumes you already have a `model` and `tokenizer` (see the loading snippet above).

<table>
  <thead>
    <tr>
      <th>
        Hugging Face (

        `transformers`

        )
      </th>

      <th>
        NeMo AutoModel (

        `nemo_automodel`

        )
      </th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>
        <pre>
          <code />
        </pre>
      </td>

      <td>
        <pre>
          <code />
        </pre>
      </td>
    </tr>
  </tbody>
</table>

### Compare Tokenizers

NeMo AutoModel provides `NeMoAutoTokenizer` as a Transformers-like auto-tokenizer with a small registry for specialized backends (and a safe fallback when no specialization is needed).

<table>
  <thead>
    <tr>
      <th>
        Hugging Face (

        `transformers`

        )
      </th>

      <th>
        NeMo AutoModel (

        `nemo_automodel`

        )
      </th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>
        <pre>
          <code />
        </pre>
      </td>

      <td>
        <pre>
          <code />
        </pre>
      </td>
    </tr>
  </tbody>
</table>

Select the loading route with `tokenizer_backend`:

* `"nemo_auto"` (the default) uses NeMo AutoModel's model-type registry and falls back to its
  Transformers-compatible wrapper.
* `"nemo_wrapped_auto"` bypasses registered NeMo tokenizers and uses Transformers `AutoTokenizer`
  with NeMo AutoModel's tokenizer compatibility wrapper.
* `"transformers_auto"` calls Transformers `AutoTokenizer` directly.
* `"tokenizers"` loads `tokenizer.json` directly through Transformers `TokenizersBackend`. Use this route
  when the serialized tokenizer should be authoritative and no repository-provided tokenizer code is needed.
  Tokenizer policy arguments such as `add_bos_token`, `add_eos_token`, `split_special_tokens`,
  `fix_mistral_regex`, and `padding_side` are forwarded to that backend.

For example:

```python
tok = NeMoAutoTokenizer.from_pretrained(
    "mistralai/Ministral-3-3B-Instruct-2512-BF16",
    tokenizer_backend="tokenizers",
    fix_mistral_regex=True,
    split_special_tokens=True,
    add_bos_token=True,
    add_eos_token=False,
)
```

The legacy `force_default=True` option is equivalent to `tokenizer_backend="nemo_wrapped_auto"`, and
`force_hf=True` is equivalent to `tokenizer_backend="transformers_auto"`.
Equivalent legacy flags and explicit routes can be combined. Contradictory route selections raise `ValueError`.

## Save Checkpoints in NeMo AutoModel and Load Everywhere

NeMo AutoModel training recipes write checkpoints as sharded safetensors by default and generate a per-checkpoint helper that can export Hugging Face-compatible consolidated safetensors after training:

* See [Checkpointing](/nemo/automodel/v0.7/development/checkpointing) for checkpoint formats and example directory layouts.
* See [Model Coverage](/nemo/automodel/v0.7/model-coverage/overview) for notes on how model support depends on the pinned Transformers version.

If your goal is to train or fine-tune in NeMo AutoModel and deploy in the Hugging Face ecosystem, keep `model_save_format: safetensors`. Use either `save_consolidated: final` for final-checkpoint export or `save_consolidated: false` plus `bash <checkpoint>/model/consolidate.sh` after training. Then load `model/consolidated/` using standard Hugging Face APIs or downstream inference engines. Set `save_consolidated: every` (or legacy `true`) only if you want inline Hugging Face export at every checkpoint save.