> This page is for version 0.7.0.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo-helix/latest/llms.txt
> - 0.8.0: https://docs.nvidia.com/nemo-helix/v0.8.0/llms.txt
> - 0.7.1: https://docs.nvidia.com/nemo-helix/v0.7.1/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo-helix/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo-helix/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo-helix/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo-helix/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo-helix/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo-helix/v0.3.0/llms.txt
> - 0.2.0: https://docs.nvidia.com/nemo-helix/v0.2.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo-helix/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo-helix/_mcp/server.

# Import and Fine-Tune Private Hugging Face Models

Use this tutorial to learn how to import a private Hugging Face model into NeMo Customizer, fine-tune it with LoRA, and deploy it for inference.

## Prerequisites

#### New to using NeMo Helix?

All platform resources—models, datasets, and more—must belong to a **workspace**. Workspaces provide organizational and authorization boundaries for your work. Within a workspace, you can optionally use **projects** to group related resources.

**If you're new to the platform**, start with the **[Setup guide](/documentation/get-started)** to learn how to deploy and evaluate models, and optimize agents using the platform end-to-end.

**If you're already familiar** with workspaces and how to upload datasets to the platform, you can proceed directly with this tutorial.

The CLI and Python SDK examples in this tutorial use the deployment and workspace in your current CLI context. To connect to a local or remote deployment, see [Connect with CLI Contexts](/documentation/reference/cli-reference/connect-to-deployments).

For more information, see [Workspaces](/documentation/get-started/core-concepts/workspaces) and [Projects](/documentation/get-started/core-concepts/projects).

### Tutorial-Specific Prerequisites

* Completed the [Quickstart](/documentation/get-started) to install and deploy NeMo Helix locally.
* Installed the Python SDK and any tutorial packages you need in your environment.
* A Hugging Face token with access to the private or gated model repository.
* A Hugging Face model with a compatible architecture. This tutorial uses `google/gemma-2-2b-it` as an example, but success depends on architectural compatibility.
* Sufficient GPU memory for the model and LoRA training job.

> **Note**
>
> This tutorial uses the current NeMo Helix SDK resources: `sdk.files.client` (the typed `FilesClient`), `sdk.models`, `sdk.inference.gateway`, and the `Customization` resource's `customization.automodel.jobs`.

### Known Issues

> **Warning**
>
> **Conv1D Model Architecture Limitation**: Models that use `Conv1D` layers are not compatible with NeMo Customizer AutoModel LoRA.
>
> **Error signature**: `AttributeError: 'Conv1D' object has no attribute 'config'`
>
> **Affected models include:**
>
> * `microsoft/DialoGPT-*` series
> * `openai-gpt` models
> * Some older `gpt2` variants
> * Other models with Conv1D-based architectures
>
> **Root cause**: These models use Conv1D layers that lack the linear layers expected by NeMo's LoRA transformation utilities.
>
> **Solution**: Use modern transformer architectures instead:
>
> * Llama models (3.1, 3.2, 3.3 series)
> * Nemotron models
> * Phi models
> * Gemma models (used in this tutorial)
>
> For a complete list of tested models, see the [Model Catalog](/documentation/customizer-reference/models/model-catalog).

## Quick Start

### 1. Initialize the SDK

The SDK needs your NeMo Helix server URL. By default, this tutorial uses `http://localhost:8080`.

```sh
export NHX_BASE_URL=<YOUR_NHX_BASE_URL>
export HF_TOKEN=<YOUR_HUGGINGFACE_TOKEN>
```

```python
import json
import os
import time
import uuid
from pathlib import Path

from IPython.display import clear_output
from nemo_automodel_plugin.schema import (
    AutomodelJobInput,
    BatchSpec,
    DatasetSpec,
    OptimizerSpec,
    OutputRequest,
    ParallelismSpec,
    ScheduleSpec,
    TrainingSpec,
)
from nemo_helix import ConflictError, NeMoHelix
from nemo_helix_plugin.client.client import NemoClient
from nemo_helix_plugin.client.errors import ConflictError as FilesConflictError
from nemo_helix_plugin.files.storage_config import HuggingfaceStorageConfig
from nemo_helix_plugin.files.types import CreateFilesetRequest
from nemo_customizer.sdk import Customization
from nemo_helix.types.secrets import HelixSecretResponse

def max_wait_time_checker(seconds: int, job_name: str = ""):
    start_time = time.time()

    def check():
        if time.time() - start_time > seconds:
            raise TimeoutError(f"{job_name} took longer than {seconds} seconds")

    return check

NHX_BASE_URL = os.environ.get("NHX_BASE_URL", "http://localhost:8080")
sdk = NeMoHelix(base_url=NHX_BASE_URL, workspace="default")
client = NemoClient(base_url=NHX_BASE_URL, workspace="default")
customization = Customization.from_client(client)
```

### 2. Store the Hugging Face Token

Private and gated Hugging Face repositories require a token. Store it as a NeMo Helix secret and reference that secret from the Hugging Face fileset.

```python
def create_or_get_secret(name: str, value: str | None, label: str) -> HelixSecretResponse:
    if not value:
        raise ValueError(f"{label} is not set")
    try:
        secret = sdk.secrets.create(
            name=name,
            workspace="default",
            value=value,
        )
        print(f"Created secret: {name}")
        return secret
    except ConflictError:
        print(f"Secret '{name}' already exists, continuing...")
        return sdk.secrets.retrieve(name=name, workspace="default")

hf_secret = create_or_get_secret("hf-token", os.getenv("HF_TOKEN"), "HF_TOKEN")
```

### 3. Create a Model FileSet and Model Entity

Create a Hugging Face-backed fileset for the private model, then register a Model Entity that points to that fileset. Model files are downloaded by the platform when training or deployment needs them.

```python
HF_REPO_ID = "google/gemma-2-2b-it"
MODEL_NAME = "gemma-2-2b-it"
model_fileset = CreateFilesetRequest(
    name=MODEL_NAME,
    description=f"Private Hugging Face model {HF_REPO_ID}",
    storage=HuggingfaceStorageConfig(
        type="huggingface",
        repo_id=HF_REPO_ID,
        repo_type="model",
        token_secret=hf_secret.name,
    ),
    cache=True,
)

try:
    base_model_fs = sdk.files.client.create_fileset(workspace="default", body=model_fileset).data()
    print(f"Created model fileset: {base_model_fs.name}")
except FilesConflictError:
    print(f"Model fileset '{MODEL_NAME}' already exists, refreshing Hugging Face settings...")
    sdk.files.client.delete_fileset(workspace="default", name=MODEL_NAME)
    base_model_fs = sdk.files.client.create_fileset(workspace="default", body=model_fileset).data()

try:
    base_model = sdk.models.create(
        workspace="default",
        name=MODEL_NAME,
        fileset=f"default/{MODEL_NAME}",
        trust_remote_code=False,
    )
    print(f"Created Model Entity: {base_model.name}")
except ConflictError:
    print(f"Model Entity '{MODEL_NAME}' already exists, updating fileset if needed...")
    base_model = sdk.models.update(
        workspace="default",
        name=MODEL_NAME,
        fileset=f"default/{MODEL_NAME}",
        trust_remote_code=False,
    )

time_check = max_wait_time_checker(600, "ModelSpec")
while not base_model.spec:
    time_check()
    time.sleep(10)
    base_model = sdk.models.retrieve(workspace="default", name=MODEL_NAME)

if base_model.spec:
    base_model.spec.linear_layers = None
    base_model = sdk.models.update(
        workspace="default",
        name=MODEL_NAME,
        spec=base_model.spec,
        verbose=True,
    )
print(base_model.model_dump_json(indent=2))
```

### 4. Prepare and Upload Training Data

Create chat-format JSONL files and upload them to a fileset. The file service, not a dataset-specific SDK resource, is the current entry point for training data.

```python
DATASET_NAME = f"{MODEL_NAME}-training-data"
DATASET_PATH = Path(DATASET_NAME).absolute()
DATASET_PATH.mkdir(parents=True, exist_ok=True)

training_data = [
    {
        "messages": [
            {"role": "user", "content": "Hello, how are you?"},
            {"role": "assistant", "content": "I'm doing well. How can I help you today?"},
        ]
    },
    {
        "messages": [
            {"role": "user", "content": "What is machine learning?"},
            {
                "role": "assistant",
                "content": "Machine learning is a way for computers to learn patterns from data.",
            },
        ]
    },
    {
        "messages": [
            {"role": "user", "content": "Can you help me with Python?"},
            {"role": "assistant", "content": "Yes. Share the code or problem you want to work on."},
        ]
    },
]

validation_data = [
    {
        "messages": [
            {"role": "user", "content": "What is artificial intelligence?"},
            {
                "role": "assistant",
                "content": "Artificial intelligence is software that performs tasks associated with human intelligence.",
            },
        ]
    },
    {
        "messages": [
            {"role": "user", "content": "Explain renewable energy."},
            {
                "role": "assistant",
                "content": "Renewable energy comes from naturally replenished sources such as sunlight, wind, and water.",
            },
        ]
    },
]

with open(DATASET_PATH / "training.jsonl", "w", encoding="utf-8") as f:
    for item in training_data:
        f.write(json.dumps(item) + "\n")

with open(DATASET_PATH / "validation.jsonl", "w", encoding="utf-8") as f:
    for item in validation_data:
        f.write(json.dumps(item) + "\n")

sdk.files.client.create_fileset(
    workspace="default",
    body=CreateFilesetRequest(
        name=DATASET_NAME,
        description="Private Hugging Face model LoRA training data",
        cache=True,
    ),
    exist_ok=True,
)
print(f"Dataset fileset ready: {DATASET_NAME}")

sdk.files.fsspec.put(
    lpath=DATASET_PATH,
    rpath=f"default/{DATASET_NAME}/",
    recursive=True,
)
print(sdk.files.list(fileset=DATASET_NAME, workspace="default"))
```

### 5. Start a LoRA Customization Job

Submit a LoRA job to the Automodel backend. Reference the model and dataset filesets using `workspace/name`.

```python
job_suffix = uuid.uuid4().hex[:4]
JOB_NAME = f"{MODEL_NAME}-lora-job-{job_suffix}"
OUTPUT_NAME = f"{MODEL_NAME}-lora-{job_suffix}"

spec = AutomodelJobInput(
    model=f"default/{base_model.name}",
    dataset=DatasetSpec(
        training=f"default/{DATASET_NAME}",
        validation=f"default/{DATASET_NAME}",
    ),
    training=TrainingSpec(
        training_type="sft",
        finetuning_type="lora",
        max_seq_length=2048,
    ),
    schedule=ScheduleSpec(epochs=1),
    batch=BatchSpec(global_batch_size=64, micro_batch_size=1),
    optimizer=OptimizerSpec(learning_rate=5e-5),
    parallelism=ParallelismSpec(
        num_gpus_per_node=1,
        num_nodes=1,
        tensor_parallel_size=1,
        pipeline_parallel_size=1,
        context_parallel_size=1,
        expert_parallel_size=1,
    ),
    output=OutputRequest(name=OUTPUT_NAME),
)

job = customization.automodel.jobs.create(
    spec=spec,
    workspace="default",
    name=JOB_NAME,
)

print(f"Submitted job: {job.job.name}")
print(f"Output adapter: {OUTPUT_NAME}")
```

### 6. Track Training Progress

Poll job status until training reaches a terminal state.

```python
time_check = max_wait_time_checker(3600, "Customization Job")

while True:
    time_check()
    status = sdk.jobs.get_status(name=job.job.name, workspace="default")
    clear_output(wait=True)
    print(f"Job status: {status.status}")

    step = max_steps = None
    training_phase = None
    for job_step in status.steps or []:
        if job_step.name == "training":
            for task in job_step.tasks or []:
                details = task.status_details if isinstance(task.status_details, dict) else {}
                step = details.get("step")
                max_steps = details.get("max_steps")
                training_phase = details.get("phase")
                break
            break

    if isinstance(step, (int, float)) and isinstance(max_steps, (int, float)) and max_steps:
        print(f"Training: step {step}/{max_steps} ({100 * step / max_steps:.1f}%)")
    if isinstance(training_phase, str):
        print(f"Phase: {training_phase}")

    if status.status in ("completed", "failed", "cancelled", "error"):
        print(f"\nJob finished: {status.status}")
        break

    time.sleep(10)

if status.status != "completed":
    raise RuntimeError(f"Training job finished with status: {status.status}")
```

### 7. Deploy the Base Model With LoRA Enabled

Create a deployment for the base model with LoRA support enabled. The LoRA adapter from training is served by the same deployment.

```python
deploy_suffix = uuid.uuid4().hex[:4]
DEPLOYMENT_CONFIG_NAME = f"private-hf-lora-cfg-{deploy_suffix}"
DEPLOYMENT_NAME = f"private-hf-lora-{deploy_suffix}"

deployment_config = sdk.inference.deployment_configs.create(
    workspace="default",
    name=DEPLOYMENT_CONFIG_NAME,
    engine="vllm",
    model_spec={
        "model_namespace": "default",
        "model_name": MODEL_NAME,
        "lora_enabled": True,
    },
    executor_config={
        "gpu": 1,
        "image_name": "vllm/vllm-openai",
        "image_tag": "v0.22.1",
        "additional_args": ["--max-lora-rank", "32"],
    },
)

deployment = sdk.inference.deployments.create(
    workspace="default",
    name=DEPLOYMENT_NAME,
    config=deployment_config.name,
)

print(f"Deployment name: {deployment.name}")
print(f"Deployment status: {deployment.status}")
```

```python
time_check = max_wait_time_checker(1800, "Deployment")
while True:
    time_check()
    time.sleep(15)
    deployment_status = sdk.inference.deployments.retrieve(
        name=deployment.name,
        workspace="default",
    )
    clear_output(wait=True)
    print(f"Deployment: {deployment.name}")
    print(f"Status: {deployment_status.status}")

    if deployment_status.status in ("RUNNING", "READY"):
        if not sdk.models.wait_for_gateway(deployment.name, workspace="default", timeout=60):
            raise RuntimeError("Inference gateway did not become ready")
        break
    if deployment_status.status in ("FAILED", "ERROR", "TERMINATED", "LOST"):
        raise RuntimeError(f"Deployment failed with status: {deployment_status.status}")
```

### 8. Test the Deployed Model

Call the inference gateway through the SDK. Use the base model name to test the original model and the output adapter name to test the LoRA-adapted model.

#### Python SDK

```python
messages = [
    {"role": "user", "content": "Can you summarize what LoRA fine-tuning does?"}
]

def chat(model_id: str):
    return sdk.inference.gateway.provider.post(
        "v1/chat/completions",
        name=deployment.name,
        workspace="default",
        body={
            "model": model_id,
            "messages": messages,
            "temperature": 0.2,
            "max_tokens": 128,
        },
    )

BASE_INFERENCE_MODEL_NAME = f"default/{MODEL_NAME}"
INFERENCE_MODEL_NAME = f"default--{OUTPUT_NAME}"
base_response = chat(BASE_INFERENCE_MODEL_NAME)
lora_response = chat(INFERENCE_MODEL_NAME)

print("Base model response:")
print(base_response["choices"][0]["message"]["content"])
print("\nLoRA-adapted model response:")
print(lora_response["choices"][0]["message"]["content"])
```

#### cURL

```bash
export OUTPUT_NAME="<output adapter name from the Python job block>"
export INFERENCE_MODEL_NAME="default--${OUTPUT_NAME}"

curl -s "${NHX_BASE_URL}/apis/inference-gateway/v2/workspaces/default/openai/-/v1/chat/completions" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "'${INFERENCE_MODEL_NAME}'",
    "messages": [
      {"role": "user", "content": "Can you summarize what LoRA fine-tuning does?"}
    ],
    "temperature": 0.2,
    "max_tokens": 128
  }' | jq
```

## Next Steps

Learn how to [check customization job metrics](/documentation/customizer-reference/tutorials/metrics) to monitor training progress and performance for your fine-tuned model.