Understanding NeMo Customizer: Models, Training, and Resources

Learn the fundamentals of how NeMo Customizer works to make informed decisions about your fine-tuning projects. This tutorial covers how models are organized, how adapters attach to base models, training types and GPU requirements, and how to choose the right approach for your use case.

Understanding these basics will help you navigate the fine-tuning process more effectively and avoid common issues. If you’re ready to start fine-tuning immediately, you can jump to SFT Customization Job after completing this tutorial.

This tutorial focuses on understanding and discovery—no actual training jobs are created.

Prerequisites

$
# Set the base URL for NeMo Platform
$
export NMP_BASE_URL="http://localhost:8080" # Or your deployed platform URL
$
$
# Optional: Weights & Biases for experiment tracking
$
export WANDB_API_KEY="<your-wandb-api-key>"

1
import os
2
from nemo_platform import NeMoPlatform
3
4
client = NeMoPlatform(
5
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
6
    workspace="default",
7
)

Core Concepts

What is a Model Entity?

A Model Entity represents a model registered in the NeMo Platform. It contains:

1. FileSet Reference: Points to the model checkpoint files (weights, config, tokenizer)
2. Model Spec: Auto-populated metadata about the model architecture (layers, parameters, etc.)
3. Adapters: LoRA or other parameter-efficient fine-tuning weights attached to this model
4. Base Model Link: Optional reference to a parent model (for fine-tuned models)

Think of a Model Entity as a “model card” that tracks everything about a model—where its files are, what architecture it uses, and what adapters have been trained for it.

What is an Adapter?

An Adapter is a set of parameter-efficient fine-tuning weights (like LoRA) that are attached to a Model Entity. Adapters:

• Are nested within the parent Model Entity
• Are enabled for inference by default post training
• Have their own FileSet for storing the adapter weights
• Track metadata like finetuning type, rank, and alpha values

What is a FileSet?

A FileSet is a collection of files stored in the platform’s file service. For customization:

• Model FileSet: Contains the base model checkpoint (config, weights, tokenizer)
• Adapter FileSet: Contains the LoRA adapter weights
• Dataset FileSet: Contains training and validation data

The Customization Workflow

Step-by-Step Breakdown

1. Create a FileSet for your base model

Upload your model checkpoint files (from HuggingFace, NGC, or local storage) to a FileSet:

1
import os
2
from nemo_platform import NeMoPlatform
3
from nemo_platform.types.files import HuggingfaceStorageConfigParam
4
5
client = NeMoPlatform(
6
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
7
    workspace="default",
8
)
9
10
# Create a FileSet from HuggingFace
11
fileset = client.files.filesets.create(
12
    workspace="default",
13
    name="llama-3-2-1b",
14
    description="Llama 3.2 1B base model",
15
    storage=HuggingfaceStorageConfigParam(
16
        type="huggingface",
17
        repo_id="meta-llama/Llama-3.2-1B-Instruct",
18
        repo_type="model",
19
        token_secret="my-hf-token",  # Secret containing HuggingFace token
20
    ),
21
)

2. Create a Model Entity pointing to the FileSet

1
model = client.models.create(
2
    workspace="default",
3
    name="llama-3-2-1b",
4
    fileset="default/llama-3-2-1b",  # Reference to the FileSet
5
    description="Llama 3.2 1B base model",
6
)
7
8
# Wait for model spec to be auto-populated
9
import time
10
11
while not model.spec:
12
    time.sleep(5)
13
    model = client.models.retrieve(workspace="default", name="llama-3-2-1b")
14
15
print(f"Model architecture: {model.spec.family}")
16
print(f"Parameters: {model.spec.base_num_parameters:,}")

3. Create a Customization Job

1
job = client.customization.jobs.create(
2
    workspace="default",
3
    name="my-email-assistant-lora",
4
    spec={
5
        "model": "default/llama-3-2-1b",
6
        "dataset": "fileset://default/email-training-data",
7
        "training": {
8
            "type": "sft",
9
            "peft": {"type": "lora", "rank": 8, "alpha": 32},
10
            "epochs": 3,
11
            "batch_size": 32,
12
        },
13
        "deployment_config": {"lora_enabled": True},
14
    },
15
)

4. Access the Result

After training completes:

1
# For LoRA jobs - adapter is attached to the model
2
model = client.models.retrieve(workspace="default", name="llama-3-2-1b")
3
4
for adapter in model.adapters:
5
    print(f"Adapter: {adapter.name}")
6
    print(f" Type: {adapter.finetuning_type}")
7
    print(f" Enabled: {adapter.enabled}")
8
    print(f" Files: {adapter.fileset}")

Understanding Adapters and Deployment

How Adapters Work

When you run a LoRA customization job:

1. Training produces adapter weights (small compared to base model)
2. Adapter created and attached to the parent Model Entity
3. FileSet created with the adapter weights
4. Enabled by default so NIMs serving the base model automatically load the adapter

Viewing Adapters on a Model

1
model = client.models.retrieve(workspace="default", name="llama-3-2-1b")
2
3
print(f"Model: {model.name}")
4
print(f"Adapters:")
5
for adapter in model.adapters or []:
6
    print(f" - {adapter.name}")
7
    print(f" Type: {adapter.finetuning_type}")
8
    print(f" Enabled: {adapter.enabled}")
9
    print(f" Created: {adapter.created_at}")

Disabling and Re-enabling Adapters

Adapters are enabled by default, but you can disable an adapter to remove it from inference without deleting it. When you set enabled=False, the sidecar running alongside the NIM automatically removes the adapter’s files on its next reconciliation pass (every few seconds). Re-enabling the adapter causes the sidecar to re-download and serve it again.

1
client.models.update_adapter(
2
    model_name="llama-3-2-1b",
3
    workspace="default",
4
    adapter_name="my-custom-lora",
5
    enabled=False,
6
)

To re-enable:

1
client.models.update_adapter(
2
    model_name="llama-3-2-1b",
3
    workspace="default",
4
    adapter_name="my-custom-lora",
5
    enabled=True,
6
)

Creating Adapters Manually

You can also create adapters manually (e.g., from externally trained weights):

1
model = client.models.create_adapter(
2
    model_name="llama-3-2-1b",
3
    workspace="default",
4
    name="my-custom-lora",
5
    fileset="default/my-lora-weights",
6
    finetuning_type="lora",
7
)

Training Types and Resource Requirements

Available Training Approaches

GPU Memory Guidelines

Storage Requirements

Customization jobs consume disk space on the platform’s shared persistent volume for model files, finetuning checkpoints, and the final output artifact. Required space depends on the training type:

For troubleshooting disk-related failures, see customizer.

Parallelism Parameters Explained

Parallelism is configured via training.parallelism. These parameters control how training workloads are distributed across GPUs:

data_parallel_size is automatically derived as total_gpus / (TP × PP × CP) and is not set directly.

Resource Allocation Rules

Training configurations must satisfy mathematical constraints to work properly:

Example Calculations:

• 8 GPUs with tensor_parallel_size=4, pipeline_parallel_size=2 ✅ Valid (8 = 4 × 2 × 1)
• 4 GPUs with tensor_parallel_size=4, pipeline_parallel_size=2 ❌ Invalid (4 ≠ 4 × 2 × 1)

Choosing Your Training Approach

Decision Framework

When to Use LoRA

✅ Choose LoRA when:

• You have limited GPU resources (1-2 GPUs)
• You want fast training iterations
• You need multiple specialized versions of the same base model
• You want to auto-deploy adapters to existing NIM deployments

When to Use Full Fine-Tuning

✅ Choose Full SFT when:

• You need maximum model performance
• You have sufficient GPU resources (4+ GPUs)
• You want complete control over all model weights
• You’re preparing a production deployment

Model Types and Capabilities

Supported Language Models

Specialized Models

Importing Custom Models

You can import any HuggingFace-compatible model:

1
from nemo_platform.types.files import HuggingfaceStorageConfigParam
2
3
# Create FileSet from HuggingFace
4
fileset = client.files.filesets.create(
5
    workspace="default",
6
    name="my-custom-model",
7
    storage=HuggingfaceStorageConfigParam(
8
        type="huggingface",
9
        repo_id="organization/model-name",
10
        repo_type="model",
11
        token_secret="my-hf-token",
12
    ),
13
)
14
15
# Create Model Entity
16
model = client.models.create(
17
    workspace="default", name="my-custom-model", fileset="default/my-custom-model"
18
)

For detailed guidance, see Import HuggingFace Model.

Next Steps

Now that you understand how Model Entities and Adapters work, you’re ready to proceed:

Key Takeaways

✅ Model Entities contain model metadata and point to FileSet with checkpoint files ✅ Adapters (LoRA) are attached to Model Entities, not stored separately ✅ FileSet is where actual model/adapter files are stored ✅ LoRA training creates an adapter on the parent Model Entity ✅ Full SFT training creates a new Model Entity with full weights ✅ Adapters are enabled by default and automatically loaded by NIMs serving the base model ✅ GPU requirements vary significantly between LoRA and full fine-tuning ✅ Custom HuggingFace models can be imported via FileSet + Model Entity

Quick Reference Commands

1
# List all Model Entities
2
models = client.models.list(workspace="default")
3
4
# Get a specific Model Entity with adapters
5
model = client.models.retrieve(workspace="default", name="llama-3-2-1b")
6
7
# Create a customization job
8
job = client.customization.jobs.create(
9
    workspace="default",
10
    name="my-job",
11
    spec={
12
        "model": "default/llama-3-2-1b",
13
        "dataset": "fileset://default/my-dataset",
14
        "training": {"type": "sft", "peft": {"type": "lora"}},
15
    },
16
)
17
18
# Add an adapter to a model
19
client.models.create_adapter(
20
    model_name="llama-3-2-1b",
21
    workspace="default",
22
    name="my-adapter",
23
    fileset="default/adapter-weights",
24
    finetuning_type="lora",
25
)

You now have the foundation to make informed decisions about your fine-tuning projects!