> This page is for version Latest (default).
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo-helix/latest/llms.txt
> - 0.8.0: https://docs.nvidia.com/nemo-helix/v0.8.0/llms.txt
> - 0.7.1: https://docs.nvidia.com/nemo-helix/v0.7.1/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo-helix/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo-helix/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo-helix/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo-helix/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo-helix/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo-helix/v0.3.0/llms.txt
> - 0.2.0: https://docs.nvidia.com/nemo-helix/v0.2.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo-helix/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo-helix/_mcp/server.

# Import and Fine-Tune Private Hugging Face Models

Use this tutorial to learn how to import a private Hugging Face model into NeMo Customizer, fine-tune it with LoRA, and deploy it for inference.

## Prerequisites

#### New to using NeMo Helix?

All platform resources—models, datasets, and more—must belong to a **workspace**. Workspaces provide organizational and authorization boundaries for your work. Within a workspace, you can optionally use **projects** to group related resources.

**If you're new to the platform**, start with the **[Setup guide](/documentation/get-started)** to learn how to deploy and evaluate models, and optimize agents using the platform end-to-end.

**If you're already familiar** with workspaces and how to upload datasets to the platform, you can proceed directly with this tutorial.

The CLI and Python SDK examples in this tutorial use the deployment and workspace in your current CLI context. To connect to a local or remote deployment, see [Connect with CLI Contexts](/documentation/reference/cli-reference/connect-to-deployments).

For more information, see [Workspaces](/documentation/get-started/core-concepts/workspaces) and [Projects](/documentation/get-started/core-concepts/projects).

### Tutorial-Specific Prerequisites

* Completed the [Quickstart](/documentation/get-started) to install and deploy NeMo Helix locally.
* Installed the Python SDK and any tutorial packages you need in your environment.
* A Hugging Face token with access to the private or gated model repository.
* A private or gated Hugging Face model with a compatible architecture. The job settings in this tutorial assume a small dense model (about 2B parameters) on one GPU; for larger or MoE models, [start from a reference recipe](/documentation/customizer-reference/manage-customization-jobs/training-configuration#reference-recipes).
* Sufficient GPU memory for the model and LoRA training job.

> **Note**
>
> This tutorial uses the typed NeMo Helix clients: `client.files`, `client.models`, and the `Customization` resource's `customization.automodel.jobs`.

### Known Issues

> **Warning**
>
> **Conv1D Model Architecture Limitation**: Models that use `Conv1D` layers are not compatible with NeMo Customizer AutoModel LoRA.
>
> **Error signature**: `AttributeError: 'Conv1D' object has no attribute 'config'`
>
> **Affected models include:**
>
> * `microsoft/DialoGPT-*` series
> * `openai-gpt` models
> * Some older `gpt2` variants
> * Other models with Conv1D-based architectures
>
> **Root cause**: These models use Conv1D layers that lack the linear layers expected by NeMo's LoRA transformation utilities.
>
> **Solution**: Use modern transformer architectures instead:
>
> * Nemotron models
> * Other architectures listed in the Model Catalog
>
> For a complete list of tested models, see the [Model Catalog](/documentation/customizer-reference/models/model-catalog).

## Quick Start

### 1. Initialize the SDK

The SDK connects to the deployment in your current CLI context. To connect to another deployment, refer to [Connect with CLI Contexts](/documentation/reference/cli-reference/connect-to-deployments). Export your Hugging Face token:

```sh
export HF_TOKEN=<YOUR_HUGGINGFACE_TOKEN>
```

```python
import json
import os
import time
import uuid
from pathlib import Path

from IPython.display import clear_output
from nemo_automodel_plugin.schema import (
    AutomodelJobInput,
    BatchSpec,
    DatasetSpec,
    OptimizerSpec,
    OutputRequest,
    ParallelismSpec,
    ScheduleSpec,
    TrainingSpec,
)
from nemo_helix_plugin.client.client import NemoClient
from nemo_helix_plugin.client.errors import ConflictError
from nemo_helix_plugin.files.client import FilesClient
from nemo_helix_plugin.files.storage_config import HuggingfaceStorageConfig
from nemo_helix_plugin.files.types import CreateFilesetRequest
from nemo_helix_plugin.models.types import CreateModelEntityRequest, UpdateModelEntityRequest
from nemo_helix_plugin.schema import SecretRef
from nemo_helix_plugin.secrets.types import HelixSecretCreateRequest, HelixSecretResponse
from nemo_customizer.sdk import Customization
from filesets import FilesetFileSystem
from pydantic import SecretStr

def max_wait_time_checker(seconds: int, job_name: str = ""):
    start_time = time.time()

    def check():
        if time.time() - start_time > seconds:
            raise TimeoutError(f"{job_name} took longer than {seconds} seconds")

    return check

client = NemoClient.from_config()
customization = Customization.from_client(client)
```

> **Note**
>
> These examples use the `default` workspace. If your current CLI context uses a different workspace, run `nemo config set --workspace default` first. To check, run `nemo config view`.

### 2. Store the Hugging Face Token

Private and gated Hugging Face repositories require a token. Store it as a NeMo Helix secret and reference that secret from the Hugging Face fileset.

```python
def create_or_get_secret(name: str, value: str | None, label: str) -> HelixSecretResponse:
    if not value:
        raise ValueError(f"{label} is not set")
    try:
        secret = client.secrets.create_secret(
            body=HelixSecretCreateRequest(name=name, value=SecretStr(value)),
        ).data()
        print(f"Created secret: {name}")
        return secret
    except ConflictError:
        print(f"Secret '{name}' already exists, continuing...")
        return client.secrets.get_secret(name=name).data()

hf_secret = create_or_get_secret("hf-token", os.getenv("HF_TOKEN"), "HF_TOKEN")
```

### 3. Create a Model FileSet and Model Entity

Create a Hugging Face-backed fileset for the private model, then register a Model Entity that points to that fileset. Model files are downloaded by the platform when training or deployment needs them.

```python
HF_REPO_ID = "<your-org>/<your-private-model>"  # Private or gated repository your token can read
MODEL_NAME = "my-private-model"
model_storage = HuggingfaceStorageConfig(
    repo_id=HF_REPO_ID,
    repo_type="model",
    token_secret=SecretRef(hf_secret.name),
)

try:
    base_model_fs = client.files.create_fileset(
        body=CreateFilesetRequest(
            name=MODEL_NAME,
            description=f"Private Hugging Face model {HF_REPO_ID}",
            storage=model_storage,
            cache=True,
        ),
    ).data()
    print(f"Created model fileset: {base_model_fs.name}")
except FilesConflictError:
    print(f"Model fileset '{MODEL_NAME}' already exists, refreshing Hugging Face settings...")
    client.files.delete_fileset(name=MODEL_NAME)
    base_model_fs = client.files.create_fileset(
        body=CreateFilesetRequest(
            name=MODEL_NAME,
            description=f"Private Hugging Face model {HF_REPO_ID}",
            storage=model_storage,
            cache=True,
        ),
    ).data()

try:
    base_model = client.models.create_model(
        body=CreateModelEntityRequest(
            name=MODEL_NAME,
            fileset=f"default/{MODEL_NAME}",
            trust_remote_code=False,
        ),
    ).data()
    print(f"Created Model Entity: {base_model.name}")
except ConflictError:
    print(f"Model Entity '{MODEL_NAME}' already exists, updating fileset if needed...")
    base_model = client.models.update_model(
        name=MODEL_NAME,
        body=UpdateModelEntityRequest(
            fileset=f"default/{MODEL_NAME}",
            trust_remote_code=False,
        ),
    ).data()

time_check = max_wait_time_checker(600, "ModelSpec")
while not base_model.spec:
    time_check()
    time.sleep(10)
    base_model = client.models.get_model(name=MODEL_NAME).data()

if base_model.spec:
    base_model.spec.linear_layers = None
    base_model = client.models.update_model(
        name=MODEL_NAME,
        body=UpdateModelEntityRequest(spec=base_model.spec),
    ).data()
print(base_model.model_dump_json(indent=2))
```

### 4. Prepare and Upload Training Data

Create chat-format JSONL files and upload them to a fileset. The file service, not a dataset-specific SDK resource, is the current entry point for training data.

```python
DATASET_NAME = f"{MODEL_NAME}-training-data"
DATASET_PATH = Path(DATASET_NAME).absolute()
DATASET_PATH.mkdir(parents=True, exist_ok=True)

training_data = [
    {
        "messages": [
            {"role": "user", "content": "Hello, how are you?"},
            {"role": "assistant", "content": "I'm doing well. How can I help you today?"},
        ]
    },
    {
        "messages": [
            {"role": "user", "content": "What is machine learning?"},
            {
                "role": "assistant",
                "content": "Machine learning is a way for computers to learn patterns from data.",
            },
        ]
    },
    {
        "messages": [
            {"role": "user", "content": "Can you help me with Python?"},
            {"role": "assistant", "content": "Yes. Share the code or problem you want to work on."},
        ]
    },
]

validation_data = [
    {
        "messages": [
            {"role": "user", "content": "What is artificial intelligence?"},
            {
                "role": "assistant",
                "content": "Artificial intelligence is software that performs tasks associated with human intelligence.",
            },
        ]
    },
    {
        "messages": [
            {"role": "user", "content": "Explain renewable energy."},
            {
                "role": "assistant",
                "content": "Renewable energy comes from naturally replenished sources such as sunlight, wind, and water.",
            },
        ]
    },
]

with open(DATASET_PATH / "training.jsonl", "w", encoding="utf-8") as f:
    for item in training_data:
        f.write(json.dumps(item) + "\n")

with open(DATASET_PATH / "validation.jsonl", "w", encoding="utf-8") as f:
    for item in validation_data:
        f.write(json.dumps(item) + "\n")

try:
    client.files.create_fileset(
        body=CreateFilesetRequest(
            name=DATASET_NAME,
            description="Private Hugging Face model LoRA training data",
            cache=True,
        ),
    ).data()
    print(f"Created dataset fileset: {DATASET_NAME}")
except ConflictError:
    print(f"Dataset fileset '{DATASET_NAME}' already exists, continuing...")

fs = FilesetFileSystem(client=FilesClient.from_client(client))
fs.put(
    lpath=DATASET_PATH,
    rpath=f"default/{DATASET_NAME}/",
    recursive=True,
)
print(client.files.list_files(name=DATASET_NAME).data())
```

### 5. Start a LoRA Customization Job

Submit a LoRA job to the Automodel backend. Reference the model and dataset filesets using `workspace/name`.

```python
job_suffix = uuid.uuid4().hex[:4]
JOB_NAME = f"{MODEL_NAME}-lora-job-{job_suffix}"
OUTPUT_NAME = f"{MODEL_NAME}-lora-{job_suffix}"

spec = AutomodelJobInput(
    model=f"default/{base_model.name}",
    dataset=DatasetSpec(
        training=f"default/{DATASET_NAME}",
        validation=f"default/{DATASET_NAME}",
    ),
    training=TrainingSpec(
        training_type="sft",
        finetuning_type="lora",
        max_seq_length=2048,
    ),
    schedule=ScheduleSpec(epochs=1),
    batch=BatchSpec(global_batch_size=64, micro_batch_size=1),
    optimizer=OptimizerSpec(learning_rate=5e-5),
    parallelism=ParallelismSpec(
        num_gpus_per_node=1,
        num_nodes=1,
        tensor_parallel_size=1,
        pipeline_parallel_size=1,
        context_parallel_size=1,
        expert_parallel_size=1,
    ),
    output=OutputRequest(name=OUTPUT_NAME),
)

job = customization.automodel.jobs.create(
    spec=spec,
    workspace="default",
    name=JOB_NAME,
)

print(f"Submitted job: {job.job.name}")
print(f"Output adapter: {OUTPUT_NAME}")
```

### 6. Track Training Progress

Poll job status until training reaches a terminal state.

```python
time_check = max_wait_time_checker(3600, "Customization Job")

while True:
    time_check()
    status = client.jobs.get_job_status(name=job.job.name, workspace="default").data()
    clear_output(wait=True)
    print(f"Job status: {status.status}")

    step = max_steps = None
    training_phase = None
    for job_step in status.steps or []:
        if job_step.name == "training":
            for task in job_step.tasks or []:
                details = task.status_details if isinstance(task.status_details, dict) else {}
                step = details.get("step")
                max_steps = details.get("max_steps")
                training_phase = details.get("phase")
                break
            break

    if isinstance(step, (int, float)) and isinstance(max_steps, (int, float)) and max_steps:
        print(f"Training: step {step}/{max_steps} ({100 * step / max_steps:.1f}%)")
    if isinstance(training_phase, str):
        print(f"Phase: {training_phase}")

    if status.status in ("completed", "failed", "cancelled", "error"):
        print(f"\nJob finished: {status.status}")
        break

    time.sleep(10)

if status.status != "completed":
    raise RuntimeError(f"Training job finished with status: {status.status}")
```

### 7. Deploy the Base Model With LoRA Enabled

Create a deployment for the base model with LoRA support enabled. The LoRA adapter from training is served by the same deployment.

```python
deploy_suffix = uuid.uuid4().hex[:4]
DEPLOYMENT_CONFIG_NAME = f"private-hf-lora-cfg-{deploy_suffix}"
DEPLOYMENT_NAME = f"private-hf-lora-{deploy_suffix}"

from nemo_helix_plugin.models.types import (
    ContainerExecutorConfig,
    CreateModelDeploymentConfigRequest,
    CreateModelDeploymentRequest,
    Engine,
    ModelDeploymentConfigModelSpec,
)

deployment_config = client.models.create_deployment_config(
    body=CreateModelDeploymentConfigRequest(
        name=DEPLOYMENT_CONFIG_NAME,
        engine=Engine.VLLM,
        model_spec=ModelDeploymentConfigModelSpec(
            model_namespace="default",
            model_name=MODEL_NAME,
            lora_enabled=True,
        ),
        executor_config=ContainerExecutorConfig(
            gpu=1,
            image_name="docker.io/vllm/vllm-openai",
            image_tag="v0.22.1",
            additional_args=["--max-lora-rank", "32"],
        ),
    ),
).data()

deployment = client.models.create_deployment(
    body=CreateModelDeploymentRequest(
        name=DEPLOYMENT_NAME,
        config=deployment_config.name,
    ),
).data()

print(f"Deployment name: {deployment.name}")
print(f"Deployment status: {deployment.status}")
```

```python
time_check = max_wait_time_checker(1800, "Deployment")
while True:
    time_check()
    time.sleep(15)
    deployment_status = client.models.get_deployment(name=deployment.name).data()
    clear_output(wait=True)
    print(f"Deployment: {deployment.name}")
    print(f"Status: {deployment_status.status}")

    if deployment_status.status in ("RUNNING", "READY"):
        if not client.models.wait_for_gateway(deployment.name, workspace="default", timeout=60):
            raise RuntimeError("Inference gateway did not become ready")
        break
    if deployment_status.status in ("FAILED", "ERROR", "TERMINATED", "LOST"):
        raise RuntimeError(f"Deployment failed with status: {deployment_status.status}")
```

### 8. Test the Deployed Model

Call the inference gateway through the SDK. Use the base model name to test the original model and the output adapter name to test the LoRA-adapted model.

#### Python SDK

```python
messages = [
    {"role": "user", "content": "Can you summarize what LoRA fine-tuning does?"}
]

from nemo_helix_plugin.inference_gateway.client import InferenceGatewayClient
from nemo_helix_plugin.inference_gateway.types import JsonBody


def chat(model_id: str):
    return InferenceGatewayClient.from_client(client).provider_post(
        name=deployment.name,
        workspace="default",
        trailing_uri="v1/chat/completions",
        body=JsonBody(
            {
                "model": model_id,
                "messages": messages,
                "temperature": 0.2,
                "max_tokens": 128,
            }
        ),
    ).data()

BASE_INFERENCE_MODEL_NAME = f"default/{MODEL_NAME}"
INFERENCE_MODEL_NAME = f"default--{OUTPUT_NAME}"
base_response = chat(BASE_INFERENCE_MODEL_NAME)
lora_response = chat(INFERENCE_MODEL_NAME)

print("Base model response:")
print(base_response["choices"][0]["message"]["content"])
print("\nLoRA-adapted model response:")
print(lora_response["choices"][0]["message"]["content"])
```

#### cURL

This example uses `NHX_BASE_URL` when it is set and falls back to `http://localhost:8080`. If your CLI context points at a remote deployment, export its URL first. Refer to [When to use `NHX_BASE_URL`](/documentation/reference/cli-reference/connect-to-deployments#when-to-use-nhx-base-url).

```bash
export OUTPUT_NAME="<output adapter name from the Python job block>"
export INFERENCE_MODEL_NAME="default--${OUTPUT_NAME}"

curl -s "${NHX_BASE_URL:-http://localhost:8080}/apis/inference-gateway/v2/workspaces/default/openai/-/v1/chat/completions" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "'${INFERENCE_MODEL_NAME}'",
    "messages": [
      {"role": "user", "content": "Can you summarize what LoRA fine-tuning does?"}
    ],
    "temperature": 0.2,
    "max_tokens": 128
  }' | jq
```

## Next Steps

Learn how to [check customization job metrics](/documentation/customizer-reference/tutorials/metrics) to monitor training progress and performance for your fine-tuned model.